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The Message is the Medium: 

Using Video Materials to Facilitate Foreign Language 
Performance 

THOMAS J. GARZA 

In connection with ongoing efforts to promote proficiency in 
communicative performance of language students, instructors of foreign 
languages and ESL/EFL continually seek to make maximum use of 
precious limited contact hours in the classroom to provide the learners 
with the most beneficial combination of language and culture in 
authentic situational contexts. This paper offers an overview of the past 
twenty years of research and practice in the application of video-based 
technology in foreign language teaching, and explores the video 
medium as a potential remedy to commonly used non-communicative 
or text-bound teaching materials. Practical recommendations for 
developing and incorporating video materials -- especially authentic 
media - into the teaching of language and culture at all levels, from 
beginning to advanced. 

Providing a Context for Communicative Performance 

Well over two decades have passed since the foreign language teaching 
community was awakened to the potential of the theory and practice of 
“communicative competence” through the works of Hymes (1968, 1972) and 
Habermas (1970) who provided, respectively, sociolinguistic and structural 
linguistic interpretations of the term. Throughout the 1980s, pedagogical 
interpretations and adaptations of “communicative” methods and materials in 
foreign language teaching — especially in English as a Second or Foreign 
Language — have developed along the lines of Hymes’ (1968) notion based on 
linguistic variability and the relationship between language and actual situational 
speech roles. He contends that communicative competence describes “what a 
speaker needs to know to communicate effectively in culturally significant 
settings” (p. 36). 

Such notions of situationally and culturally appropriate “performance” in a 
foreign language were of paramount importance during the nascence of the 
proficiency movement and remain the foundation of many performance-based 
curricula and materials at present. In short, to address communicative competence 
in the classroom, foreign language teachers have become acutely aware of a 
universal observation: students require more than good grammar skills and broad 
vocabularies to make them proficient participants in authentic foreign language 
situations. Indeed, many students know the frustration of trying to master a 
foreign language to near-textbook perfection, memorizing interminable lists of 
lexicon and irregular verb conjugations, declensional paradigms, rules of syntax, 
and spending hours in the language lab repeating minimal pair drills, only to 
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arrive finally in the foreign country to encounter a din of unintelligible 
utterances, an array of unfamiliar social situations, and be unable to report a 
problem to the hotel manager or to place long-distance phone call through an 
operator. Stevick (1983) offers the following amusing, yet accurate, example of 
a non-communicative exchange between a native speaker of English and an ESL 
student: 

• You (with your arms full of books): *Do you think you could 

open that door?’ 

• Student (without moving to the door): Yes, l do * 

The student in Stevick’s example clearly understands the individual linguistic 
elements of the utterance of the native speaker of English and, in turn, produces 
a grammatically correct response. Yet this level of understanding and response, 
while linguistically accurate, fails in terms of communicative performance. 

For the purposes of this discussion, the term “communicative performance” 
describes the student’s ability to comprehend and produce language appropriate 
for a given situation so that all utterances are understood by all participants in 
the exchange. Such a definition implies tenets of sociolinguistics and discourse 
analysis which run contrary to purely structural/grammatical approaches to 
language learning, sometimes even opting for a “nonstandard” (albeit correct!) 
usage of language in particular communicative situations. As students and 
teachers alike continue to voice the need to develop active listening and speaking 
skills — the foundation of communicative performance — the search of methods 
and materials of instruction to address this need continues to intensify. 

Emerging quickly from this search was the realization that, even in the very 
best of the so-called “communicative” textbooks and materials of the 1980s, 
there was a serious dearth of authentic speech models with which the student 
could practice both aural comprehension and active oral production in the foreign 
language. “Authentic” speech models is understood here as examples of 
situational language use by native speakers as understood by other native 
speakers. Further, when trying to present authentic language situations for study, 
printed texts with still photographs and illustrations (the principal classroom 
interface for most of our students in secondary school- and university-level 
language programs), even with the addition of audio tape recordings, still lack 
the dynamic quality necessary to provide an active context to present and work 
through the various paralinguistic features of the language being used. These 
features, such as register, tone, gesture, proxemics, etc. are often as essential for 
the successful completion of the communicative act as is the use of the correct 
form of a verb. 

Only since the introduction of video-based materials into the foreign language 
classroom in the mid-1970s (around the same time that the term "communicative 
competence” came into vogue!) have teachers and students had available to them 
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the raw material necessary to simulate or replicate a real-life immersion 
experience in both the target language and the target culture. As Lonergan (1984) 
maintained in the early days of video use in the foreign language classroom: 

Video brings a slice of real life into the classroom. It presents the 
complete communicative situation. Language learners not only hear the 
dialogue, they also see the participants in the surroundings where the 
communication takes place. This visual information not only leads to a 
fuller comprehension of the spoken language, but can also benefit 
learners in a number of other ways. 

Thus, with the entire communicative situation demonstrated within the context 
of the video segment, much of the linguistic material, such as lexical meaning 
and usage, is made clear to the student without a formal dictionary definition. 
Similarly, social relationships and inherent behavior are contextualized visually 
so as to clarify intangible concepts such as emotion, disposition, demeanor and 
tone. 

Indeed, sociolinguistic elements permeate virtually all authentic video 
materials and carefully selected authentic video media offer not only 
contextualized situations of language use, but provide the added benefits of 
visually-conveyed information on both linguistic and cultural meaning. 
Vereshchagin and Kostomarov (1990), Russian specialists in language and 
culture contend: 

Facts perceived visually become the personal experience of the student, 
while verbal explanations reflect a detached foreign experience; not 
without reason is it said that it's better to see something once than to 
hear about it a hundred times. Besides, visual perception usually cannot 
be replaced by words. Therefore, the role of the visual mode remains 
unlimited, unique. Finally, the information input capability of the 
visual perception mode is almost ten times greater than the audio; 
therefore, the former is more economical than the latter. 

Recently, researchers and specialists such as Kramsch (1993), not only concur 
with this Soviet perspective, but expand on it, indicating the unique suitability 
of selected authentic video materials as a foreign language teaching medium, 
citing the addition of the visual modality and "slice-of-real-life" quality of good 
video as a significant contributor to total holistic comprehension. 

In discussing the teaching of lingua-culture in a foreign language, or the 
synthesis of foreign language and culture, the authentic nature of instructional 
materials - especially video - is of paramount importance. Materials prepared 
for native speakers of a language are by virtue of their intended audience saturated 
with imbedded cultural references that depend on the shared prior text of that 
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audience. Standard television programming, feature films, commercials, 
documentaries, and news items can all serve as effective pedagogical source 
materials for teaching foreign linguoculture on all levels. Because video allows 
for both audio and visual modalities of information input, the language and 
cultural material is more readily contextualized and, thus, more accessible to the 
learner. 

For purposes of exploiting the cultural as well as the linguistic material of the 
video segment, "good" foreign language video should be multi-layered, 
incorporating current and useful situational language, visually supported 
paralinguistic elements, and inherent cultural content. It is the presence of such 
culturally-bound material that, while self-evident to the native speaker of the 
target language, prevents the learner from enjoying complete, "native-like" 
comprehension of the segment, even when the language of the segment is fully 
understood. 

As with audio tape recordings, videotapes can be stopped and reviewed at the 
instructor’s or learner’s discretion in order to review a certain part of the material. 
But, unlike audiotape, videotape also allows any single frame or moment of 
action to be “frozen” on-screen for analysis or commentary. Videotape can also 
be mechanically edited, altered or manipulated during its production to enhance or 
modify the content of a segment to make it more conducive to the more purpose- 
specific tasks of language instruction. 

Today, the universal availability and modest cost of standard video playback 
equipment makes it a sine qua non for the proficiency and performance based 
language classroom. For around four hundred dollars, any language program can 
be equipped with a video player and monitor to allow for the incorporation of 
video materials into the basic curriculum and syllabus for instruction. Having 
made the decision, however, to utilize the rich and varied resources of video 
materials in foreign language instruction, a myriad of related issues and questions 
emerges on the most efficient and effective use of such materials, beginning with 
the issue of selecting the most appropriate materials from among the seemingly 
unlimited amount of television, documentary, film, and commercial material 
currently available in many languages on video. 

Basic Criteria for Instructional Video 

As with selecting all instructional materials, there is good video and bad video 
for language teaching purposes. The basic criteria for “good” or useful 
instructional video emerge after considering the inherent needs of the foreign 
language classroom. In selecting or adapting video materials for language 
teaching, it is essential that these pedagogical considerations are not subordinated 
to the medium. As Willis (1983) notes, “...in the excitement of experimenting 
with a relatively new medium there is a tendency for video viewers (both us, as 
teachers, and students themselves) to lose sight of language teaching objectives. 
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and of students’ own learning objectives, and to use video for insufficiently 
motivated purposes” (p. 43). 

First, useful video must contain the desired linguistic material for instructional 
purposes. In most cases, for language courses attempting to develop 
communicative performance, this criterion means language that is current, useful 
and accurate in a corresponding situation. The video selector may find it useful to 
prepare in advance a comprehensive checklist of various linguistic features 
(specific lexicon, syntactic structures, etc.) and language functions (greetings, 
leave-takings, introductions, requests, instructions, etc.) which s/he plans to 
cover in the course and gauge the appropriateness and utility of various video 
segments by this list. 

Second, the video segment should be thematically interesting. This point may 
seem obvious at first blush, but actually entails a good deal of consideration on 
the part of the video selector. S/he must consider a variety of relevant variables, 
such as the constituency of the target audience (breakdown of age, native 
language(s)), proficiency level, and goal of language study, before attempting to 
select interesting video segments for a given course. Taking into account such 
factors, the selector is able to establish general thematic parameters for a specific 
language program. The goal of this criterion is to produce video materials that 
deal with subjects in which the students can best identify with the characters and 
the situations depicted on the screen. 

A related sub-point of the second criterion concerns the actual quality or 
production value of the video segment. The segment itself should present the 
viewer with some kind of discreet story, with an inherent beginning, middle and 
end, whether it is dramatic piece, a documentary, or an instructional “how-to” 
segment. Such a storyline is essential for maintaining student interest during the 
repeated viewings of the video segment and, thus, the successful treatment of the 
various topics covered in the video-based course. A natural storyline also allows 
for comprehension exercise potential based on the segment context as discussed 
below. 

Third, good video materials for foreign language instruction are multi-layered; 
that is, they require (or at least encourage) repeated viewings in order for the 
student to comprehend fully the content of the segment. The various “layers” 
may include linguistic and paralinguistic material, dramatic elements, depictions 
of relationships, and cultural information. Each of these textual layers may be 
addressed individually or in combinations through exercises designed to present, 
review and practice (produce) the material in the video segment. Lonergan (1984) 
contends that this capacity for repeated viewing is essential for the successful use 
of instructional video (p. 32). 

Fourth, the ideal video segment for the language classroom has a high 
audio/visual correlation. As mentioned above, the visual element should enhance 
and clarify the text by contextualizing the language nonverbally. Thus, the 
language video should provide visual images which closely parallel and support 
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the text of the audio track. In this connection, interviews and other “talking 
head” commentaries, in spite of their often provocative and engaging thematics, 
usually make unsatisfactory video materials for language instruction, since they 
provide little more communicative information than an audio tape recording 
(Geddes, 1982). 

Finally, there is the issue of length of the video segments themselves. Because 
techniques of using video in foreign language teaching are, for the most part, 
still in the test-teaching stages, data concerning the “ideal” length of video 
segments is still being gathered and analyzed. Still, existing research has 
provided some general parameters for determining segment length. One early 
study of video use in the EFL classroom documented teacher preferences for 
segments between two and ten minutes in length, stating that longer segments 
provided too much material to cover or handle in lesson format, whereas 
segments shorter than two minutes were too brief to establish any kind of 
meaningful storyline (MacKnight, 1981). More recent data collected at Harvard 
University indicate that useful discreet segments may be as brief as thirty 
seconds, but found that students’ retention of detail -- even for advanced learners 
- diminished significantly after approximately seven minutes of viewing, 
depending on the complexity of the video material. British video specialist 
Lavery (1984) is even more categorical in delineating the length of a useful video 
segment for language teaching: [for a one-hour class - 3-4 minutes]. Basically, 
instructional video should be kept to concise and discreet segments which allow 
for thorough presentation and coverage of the linguistic and paralinguistic 
information in the footage. In some cases, a long segment may be divided into 
two or more segments for more complete classroom treatment of the material. 
Or, conversely, several short, thematically related segments may be combined 
into a useful montage. The amount of actual language and visual information in 
the segment should be the determining measure in editing down the final usable 
footage for the classroom. 

Currently, video materials for the foreign language classroom come from three 
primary sources: 1) off-air television broadcast (Maxwell, 1983), 2) 

commercially available tapes or disc (non-language related), and 3) specially 
produced foreign language materials including both commercial and teacher-made 
videos (Silva, 1984). Inherent to all three options is one of the most 
controversial issues currently surrounding the use of video-based materials: Is it 
better to use video materials that are specifically designed for language 
instruction, or to adapt authentic materials for classroom use? Both alternatives 
have pros and cons associated with their adoption in a program of foreign 
language study. 

Interestingly, virtually all commercially-available British and American 
foreign language video programs for teaching ESL and EFL have opted for using 
specially produced footage. The advantages of producing all video footage oneself 
can be summarized in one word: control. The author of produced video materials 
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has complete control over some of the most important aspects of a video 
language course, most notably the language itself. The author(s) of the video 
script may devise and design entire lessons which focus on specific lexicon, 
grammatical structures, or themes within the context of a planned and structure 
storyline. Similarly, self-produced video projects allow for complete control of 
character and situational development, and, as a result, are able to develop 
supporting text and exercise materials simultaneously with the video segments. 

Paradoxically, the main disadvantage to producing new footage for a language 
program stems directly from the linguistic, situational, and production control of 
such a project. By entirely scripting the language, situations, and characters of a 
given segment, the authenticity of all three elements is threatened. This danger is 
heightened by attempts of the well-intentioned language teacher to address the 
particular needs of the audience — all in one video segment! The frequent result 
is a video segment depicting unbelievable characters in an improbable situation 
using contrived and overly prescriptive language. 

The alternative to producing new video segments for a language program is 
use authentic materials and existing video footage, which may be taken from 
feature films, documentaries, television programming (dramas, comedies, game 
shows, news, etc.), commercials, travelogues, animated segments, or any other 
video source and “repurposing” — or adapting — them for educational use. Such 
videotaped materials have the tremendous communicative benefit of having been 
originally produced for a native audience (Kerridge, 1982). Thus, even “scripted” 
texts, such as motion picture and television scripts, are written to convey 
relevant situations, depict believable characters, and use authentic language. 
Their original goal is to persuade, entertain, inform or evoke emotion in native 
speakers of the language; as such, these texts can provide students with 
engaging and functionally rich sources of authentic and highly contextualized 
language and cultural information. 

The obvious disadvantage related to the choice of authentic video footage is the 
limited control of the linguistic content of the segments. In order to obtain 
footage that will cover the range of linguistic needs of a given language course, 
the teacher or curriculum developer may have to screen a sizable amount and 
variety of material. However, this is primarily a time and personnel 
consideration; virtually any country with video technology has no end to the 
amount of existing footage from which to select language teaching materials. 

The ultimate obtainability of existing footage for a language program - 
whether recorded off-air or purchased from a distributor - depends on securing 
rights and clearances from the original producer to use the segment in an 
educational program. This consideration raises another disadvantage to using 
found footage. In the case of off-air recording for educational use, the law is still 
problematically vague and in a constant state of flux. In legal terms, teachers 
seem to fall into a gray area between the rights of a home-video user and a video 
pirate. As of this writing, it is generally agreed that educators may record 
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television programming off-air and use that material in a classroom for up to a 
period of forty days from the program’s air date, after which time the taped 
program must be destroyed (Altman, 1989). Such a practice precludes the 
assembly of a video library for long-term use in a foreign language program. 
Thus, the teacher must prepare each lesson anew and design materials to 
accompany new videos virtually every month of every year s/he uses them. 

For materials developers or educators attempting to assemble and purchase 
authentic materials for a program, the denial of rights by many video sources 
may prove to be a significant obstacle in assembling a group of suitable 
segments. In addition, the securing of available rights can be relatively 
expensive, even compared to the average cost of producing good quality original 
footage specifically for the classroom. The apparent communicative advantages 
of authentic video materials must be weighed against the legal and cost issues of 
obtaining such footage for foreign language teaching. 

Exploiting Video Materials for Language Teaching and Learning 

Though the term sometimes seems to be read somewhat aggressively in the 
United States, “video exploitation" has been a part of the British research and 
literature on video in language teaching for the past decade. The term quite 
simply refers to the structuring of activities, tasks or exercises, which explain, 
develop, or expand the communicative information contained in a video segment. 
It is at the exploitation level that the potential of video as a means of addressing 
communicative performance in foreign language teaching is greatest. Since each 
piece of footage has its own particular “character” in terms of images and 
language, it is essential that appropriate kinds of exploitation be devised for each 
case. 

The first requisite condition for successful implementation and exploitation of 
video-based materials in the foreign language classroom is to change the viewer’s 
attitude toward watching visual media from predominately passively to 
overwhelmingly active. The term “active viewing” is essential to the 
metalanguage of video in language teaching. Active viewing can be encouraged 
at various levels of designing and using video materials . At the 
design/production level of assembling video materials, several exploitation 
techniques can be employed to facilitate active viewing of the segment. 

• Captioning One of the most successful, yet seldom applied, of these 
processes is captioning of the video material. Captioning refers to the addition of 
subtitles — in the same language as the audio track - to a video segment 
(Parlatto, 1986). That is, a segment from a French film would have French 
subtitles, an American television would have English subtitles, etc. Captioned 
material appears to the viewer/leamer on the screen simultaneously as s/he hears 
the words spoken. 
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Two types of captioning exist in broadcast television programming: closed 
and open. Closed captioning, familiar to most television viewers by the symbol 
“CC” or 44 ” in program listings, is done by the National Captioning Institute 
for the hearing impaired or one of several private companies which is equipped to 
perform this service. A special decoding device must be attached to the television 
set in order to receive the printed text along the bottom of the screen. 
Televisions produced in the U.S. after 1991 often come with such a decoder 
already factory-installed. With a caption decoder and a VCR, closed-captioned 
programming can be recorded with captions onto a blank tape. Thus, captioned 
off-air television broadcasts can be prepared for use in the classroom at a later 
time and previously closed-captioned segments are converted into open-captioned 
materials. 

Open captioning or “reverse subtitling” refers to the use of captions on the 
screen which appear as part of the original broadcast and, thus, require no special 
equipment to receive or decode them. In the U.S., very few programs are 
currently aired using open captioning, except for some public service 
announcements and educational programs produced by the PBS. However, in 
addition to the above-mentioned technique of recording off-air closed-captioned 
broadcasts, open captioning could certainly be employed in the production of 
either original educational videos or in the editing and assembly of authentic 
found footage. 

Open captioning used as video exploitation can be adapted according to the 
nature of the segment and the goals of the related tasks. There are three options 
for open captions: 1) verbatim, 2) key words in context, and 3) cloze. Verbatim 
captioning is possible only when the amount and rate of speech allows for the 
transcription of each word to appear on the screen for an adequate amount of 
time. It is particularly useful for segments which aim to develop lexicon and 
expand grammatical structures. Key words or phrases captioning is recommended 
when the related segment task is to check for general comprehension of the 
language, or global meaning of the footage. Cloze captioning, in which certain 
elements of the script are eliminated from the captions and replaced with blanks, 
is used when the task of the viewer is to listen for specific information, such as 
proper names, numbers, certain phonemes or sounds, etc. 

An early study conducted by Price (1983) at Harvard University demonstrated 
that ESL students using captioned and non-captioned video materials “benefited 
significantly from captioning even with only one viewing” (p. 1). More 
specifically, the Harvard study tested 450 EFL students in six language 
competency areas based on the viewing of four different video segments. Half of 
the group watched the segments with captions, half viewed them without. All of 
the segments used NCI “closed” captions, which frequently simplify lexically 
and syntactically difficult language; however, three of the four segments shown 
in this study maintained verbatim or near verbatim captioning. Over a series of 
variables controlling for factors such as fatigue, sequence of presentation, and 
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interest value, the results across the various competency areas showed 
statistically higher scores for students viewing the segments with captions. 

More recently, in a 1989 study conducted at the National Foreign Language 
Center, Garza (1991) demonstrates a significant level of generalizability of ESL 
results in captioning studies with other foreign languages, such as Russian. The 
most significant conclusion suggested by this study is that captioning may help 
teachers and students of a foreign language bridge the often sizable gap between 
the attainment of proficiency in reading comprehension and in listening 
comprehension, the latter usually lagging significantly behind the former. By 
providing students with a familiar (i.e„ comprehensible) graphic representation 
of an utterance, they are empowered to begin to assign meaning to previously 
unintelligible aural entities, gradually building their aural comprehension in 
relation to their reading comprehension. For example, a student of ESL might 
view a video clip showing two students meeting after class. One says to the 
other (in the aural text), "Djeetjet?" to which the interlocutor replies, "No, dju?" 
The non-native student of English is at a loss to assign meaning to the 
utterances in this everyday conversation. But when the captions show, "Did you 
eat yet?" "No, did you?" as the underlying language of the spoken text, the ESL 
student quickly understands and makes the connection between the familiar 
printed text and the now comprehensible aural text. Like learning a new 
vocabulary item when reading, the student working with captions will likely not 
miss the aural cue of a captioned expression the next time s/he encounters it in 
speech. The data collected in this study all indicate that captions enhance the 
learning of a foreign language by: 1) increasing the accessibility of the salient 
language of authentic video materials, giving student the opportunity to 
understand and enjoy the same types of linguistic input understood by a native 
speaker of the language; 2) allowing the student to use multiple language 
processing strategies to accommodate the multiple modalities of input when 
captions are used; 3) increasing the memorability of the essential language and 
thus, 4) promoting the use of new lexicon and phrases in an appropriate 
context. 

• Colorization Related to the addition of open captions to video-based 
materials is the use of color highlighting of various elements of the captioning 
on the screen, such as specific lexicon, idioms, grammatical forms, or 
phonemes. Such a technique focuses the viewer’s attention on specific linguistic 
features and allows for interface with additional exercise material during the 
lesson. Throughout the 1980s, a popular PBS program entitled 
“ColorSounds®,” used open captioning of contemporary music videos with 
color highlighting of both grammatical items and phonemes (Bell, 1984). This 
program was aimed at developing literacy among American minority youth, but 
was also used successfully in the EFL classroom to teach pronunciation and 
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speaking skills (Garza, 1984). This video technique adds another “layer” to the 
instructional video to promote meaningful, repeated viewing. 

• Video Enhancement and Overlay Another video exploitation 
technique which aids in making the visual medium more conducive to presenting 
and teaching language materials is video enhancement or overlay. Such 
enhancement encompasses a variety of video “tricks” such as highlighting or 
colorizing parts of the video image itself for emphasis, adding or deleting parts 
of the video image as part of an exercise, etc. Graphics, text or a combination of 
the two may be used similarly to exploit a communicative feature of the 
segment. For example, a speech balloon — as in a comic strip — can be 
superimposed on the video image to allow students to supply appropriate speech. 
Video enhancement, like most open captioning, is a production technique added 
to video footage as part of originally-produced footage, or is added to authentic 
footage during editing and assembling for classroom use. 

• Time Code Finally, most commercially-available language video 
programs have some kind of on-screen reference code, such as a time clock or 
counter, which serves two main purposes: 1) to provide a kind of video 
bookmark which allows the viewer to stop the videotape in mid-segment and 
later return to the same place, and 2) to allow for an accurate interface with 
textbook or in-class exercises/activities and specific moments in the video 
segment. 

• Printed Materials The need for an on-screen video reference code is 
essential when video exploitation includes a textbook or viewing guide, as most 
commercial language video programs do. Printed materials can help to organize 
and structure the student’s viewing of any video materials, including off-air or 
other authentic video segments recorded by the teacher. Language video 
specialists agree on three basic categories for printed exercise materials to 
accompany a video segment: preview, task viewing, and follow-up (Stempleski, 
1990). 

Preview exercises make the video segment more accessible and less 
frustrating for the student by introducing and reviewing new or difficult material, 
such as lexicon, grammatical constructions, speech functions and general 
thematics. These exercises attempt to make the viewing process more 
immediately useful and enjoyable for the student. They should not, however, 
make the viewing of the segment superfluous by revealing the storyline, new 
information, or the humor/suspense in advance. Because the ultimate goal of 
such exercises is to prepare the student to view a segment, art, drawings, 
diagrams, realia (photographs of actual relevant items from the target culture, 
such as menus, newspapers, advertisements, tickets, etc.) or any other visual 
reference is extremely useful. Preview exercises may take a variety of forms: a 
brief reading passage, diagram fill-in, matching items, problem solving. 
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Regardless of form, it is important that the preview exercise still requires the 
learner to process and use the relevant language, not merely provide a list or 
chart of necessary information. Alternatively, a preview exercise may use a 
format that is related to the actual video segment, but does not actually occur as 
part of the segment. 

Task viewing exercises are designed to focus the student’s attention on each 
of the many layers — linguistic, structural, functional, and cultural - of the video 
segment. The activities of the task should, as much as possible, require the 
viewer to watch and listen for information that would be relevant in real-life 
situations. That is, we read movie schedules to find out what films are playing 
in which theaters, and times the showings are, or we listen to a weather report to 
find out if we need to take an umbrella to work or not. Similarly, task viewing 
exercises maintain the authentic listening and viewing purposes of the real-life 
situations depicted in the video. It is at this level of text interaction that the 
discrete storyline of a video segment may be exploited as a comprehension task. 
The tasks may progress from focusing on the gist of a segment to finer details as 
the student views and re-views it. The student provides the missing parts of the 
story - beginning, middle, or end - using contextual clues from the segment. 

The term “task viewing” is not an arbitrary one for this kind of exercise. They 
are teaching devices, not testing devices. They are to aid the students’ 
understanding, not test their memories. For example, a set of true/false 
statements given immediately after a viewing tests memory; the same exercise 
given before viewing can help students organize their viewing so that they listen 
for specific information. Thus, the relationship between the preview and task 
viewing exercises is essential to the guided viewing of the segment by the 
student. The task viewing exercise may utilize charts, lists, statements, etc. that 
help the student organize the material in the segment , or have the student relate 
or associate one level of material (specific language, gestures, facial expressions) 
in the segment with another (situation, emotion, attitude). 

Follow-up exercises are intended to help the viewer see the broader 
application of what s/he has understood in the video segment. Thus, if the video 
segment presented an aerobics instructor using various imperative forms, the 
follow-up exercise could provide more work on the imperative in a different 
context, such as in a how-to cooking lesson. Follow-up exercises are also useful 
to show varying degrees of register or tone in conversational exchanges similar 
to those depicted in the video segment. Ideally, follow-up exercises add to the 
layers of information presented in the video segment by building on the same 
linguistic, thematic, and cultural lines. 

All three types of textbook interfaces with the video materials - preview, task 
viewing, and follow-up - serve to clarify and exploit the content of the segment 
and make it useful as a language and situational model for the student [See 
Appendix A for a summary presentation of all three types of activities]. 
Together, video and text comprise a potentially powerful team in the foreign 
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language classroom, especially toward the goal of developing proficiency in 
communicative performance. With continued research, designing, and test 
teaching of such materials, video-based language programs may become the 
standard of proficiency-oriented instruction in our classrooms. In the meantime, 
it is essential that both teachers and materials developers examine carefully 
existing video programs for language teaching, as well as their own role in 
preparing and using such materials in order to asses where we can go from here. 
The video program does not yet exist that can bring a student from linguistic 
competence to communicative competence. The role of the instructor as viewing 
moderator is crucial to the success of all existing programs and must not be 
subordinated to the video medium itself. 

Practical Considerations of Video in Today’s Classroom 

The most common issue raised by practitioners who are considering adopting 
the use of video materials in their own foreign language classes concerns the 
efficacy devoting additional time to select and edit the video materials 
themselves, and then to prepare useful materials for exploitation of the 
segments, both in class and out. Given the enormous demands on the 
instructor’s time already cutting deep into any additional time for materials 
development, the use of video often appears to be more of a luxury or even a 
burden for many instructors. Such considerations, combined with the growing 
number of programs - especially at the secondary school levels - which require 
the teacher to prepare materials for and teach several levels of the same language 
in any given day, seem far to outweigh any possible benefits that could come as 
a result of using video. Simply stated, it often appears to be far too time 
consuming to create new video materials for teaching multiple levels of a 
language at once. And yet, the very flexible nature of the medium itself, together 
with the inherent richness of a good, well-selected segment, allows for an easy 
solution: one good video segment may be exploited for use at the beginning, 
intermediate and advanced levels of language training. 

In order to discuss the multi-level exploitation of video materials, it is 
necessary to invoke the Proficiency Guidelines of the American Council on the 
Teaching of Foreign Languages (ACTFL, 1986). These Guidelines establish 
descriptors for assessing the proficiency level of learners in all four language 
skills. The question as to whether or not these Guidelines should have an impact 
on how or what we teach in the classroom has sparked debate since the adoption 
of the Provisional Guidelines in 1982. Some language assessment specialists, 
such as Higgs (1987), contend that the mere existence of criteria for proficiency 
assessment does not necessarily make any recommendation on either syllabus or 
methods, stating that the Guidelines themselves are explicitly pedagogy 
independent and inform neither methodology nor syllabus decisions. In spite of 
such objections, however, the vast majority of the foreign language teaching 
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community at-large not only perceive the Guidelines as relevant to actual 
classroom instruction, but have accepted such terminology a "proficiency-based 
instruction" and "proficiency-oriented materials" in the profession as standard 
usage. 

Germane to using video materials in foreign language teaching is the 
utilization of the ACTFL skill level descriptors to assist instructors in selecting 
or creating exploitation tasks that are appropriate for the particular level of the 
students. This approach reinforces the notion that most video materials -- 
especially from authentic sources - are level independent as foreign language 
text; any good video text contains material that is accessible to learners at all 
levels, from novice to superior. Of course, more advanced learners are able to 
glean more meaning from the text more quickly than novice learners are, but 
even rank beginners possess the ability to identify cognates and make visual 
identifications. The key rule to producing video exploitation materials is to 
adjust the task, not the text. 

To this end, the Generic Descriptions for Listening in the ACTFL Guidelines 
specify at the Novice-Mid level that the learner is: 

Able to understand some short, learned utterances, particularly where 
context strongly supports understanding and speech is clearly audible. 
Comprehends some words and phrases for simple questions, statements, 
high-frequency commands and courtesy formulae about topics that refer 
to basic personal information or the immediate physical setting. The 
listener requires long pauses for assimilation and periodically requests 
repetition and/or a slower rate of speech. 

The video materials designer may now sketch out the types of tasks and 
activities that could be reasonably expected of a learner performing at the novice- 
mid level, such as: 

1) Guess the [Russian, French, etc.] cognates for English words 

provided. 

2) Identify these cognates in the actual video text. 

3) Make assumptions about the story from visual clues. 

4) Use LI text to get background information about a new cultural 
entity. 

5) Identify any questions asked in the video text. 

Based on such an outline, appropriate Preview, Task Viewing and Follow-Up 
tasks can be devised to make the video material accessible to learners at their 
level of performance. Similar types of materials can be devised at all levels of 
proficiency. 
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Conclusion 

Throughout the literature on video applications in foreign language learning 
and teaching, two tenets are consistently maintained: 1) the technological 
capabilities of video are well suited to provide a stimulating and highly engaging 
learning experience for language students, and 2) properly selected and/or 
produced video materials can present the necessary elements (authentic language 
and situations, native participants, cultural references, contextualized speech 
models, etc.) for effective proficiency-oriented instruction. 

And yet, examples of effective video courseware and programs addressing 
communicative performance are relatively few and often exist only as parts or 
constituent elements of a larger, more traditionally text-based course. In the 
university setting, programs such as Destinos for Spanish instruction and French 
in Action for French are rare examples of successful video-based courses for 
foreign language. With so many specialists and practitioners in agreement on the 
potential of this technology, why has so little of this potential been put into 
service? Perhaps the shortcomings of existing video-based programs and the 
reluctance of the language teaching community to embrace them stem directly 
from a preoccupation of their developers on the motivational and stimulating 
capabilities of the medium, while subordinating the quality of the program 
content. While it is essential to realize the technological capabilities of the 
hardware that will execute the program, it is more important to remember that 
the video segments themselves contain the necessary linguistic and paralinguistic 
information for the student interested in attaining proficiency in the language and 
culture. 

The last thirty years of foreign language instruction in this country have 
witnessed the initial popularization and eventual “demise” of two major 
instructional technologies: the audio-based language laboratory and computer- 
based language learning 1. In both cases, the relevant technology was touted as 
possessing the potential for addressing serious issues in language teaching and 
learning that could not easily and/or effectively be handled in the traditional 
classroom. When neither innovation brought about the impressive 
improvements expected, both were quickly dismissed, leaving large numbers of 
tape carousels and personal computers sitting idle. Interestingly, though, the tape 
machines and the computers performed as promised throughout their use; the 
technology never failed to deliver what it had originally promised language 
teachers: potential. The technology had not failed, the materials had. The listen- 



1 Computer-based instruction here refers to the technology and materials that 
existed prior to the development of multimedia platforms for micro computers 
since 1990. Needless to say, the newer media-based machines are of great value 
to instructors wishing to incorporate video and other media into their teaching, 
especially on an individualized basis. 
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and-repeat sequences, the rote drill-and-practice exercises, and the electronic page- 
turners did not utilize the potential of their media. 

The potential of the most recent technology, video, has understandably been 
viewed skeptically by some. To an even greater degree than the language lab or 
the computer, video offers an attractive, though somewhat deceptive, alternative 
to writing long lesson plans and conducting tedious conversation sections for 
language classes. The uninitiated teacher may choose simply to play a videotape 
and saturate the students with “comprehensible input” or “listening 
comprehension” practice. Such an approach exploits video as little more than a 
low-tech babysitter from which neither student nor teacher profits. Given the 
current state of video technology, instructors and materials developers need never 
resort to such measures. Unlike its technological predecessors in language 
teaching, video offers the instructor almost complete control in the selection, 
design, production and execution of the source materials. Instructors can create an 
instructional package of video and exploitation materials tailored to the needs, 
goals and constituency of any given class that actually exploits the potential of 
this powerful medium, particularly as a facilitator of communicative performance 
in a foreign language. 
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APPENDIX A: Text Exploitation of Video Materials 

1. Preview 

Makes the material (linguistic and non-linguistic) of the video 
segment more readily accessible to the learner by: 

• Introducing new concepts (lexical, grammatical, functional, cultural, 

etc.) before the first viewing of the segment; 

• Providing background information to help learner develop native-like 

schemata or "prior text" to understand video material (basis for 
cultural literacy); 

• Allowing learner to apply native language strategies to new material; 

• Preparing the learner to comprehend the material without giving away 

the "punch" of the segment. 

2 . Task Viemng 

Guides the learner in "peeling" away the various layers of the video 
segment and discover and master the linguistic, paralinguistic, and 
cultural material contained in it by: 

• Requiring the learner to view and re-view the video material 

in order to solve the assigned task; 

• Focusing the learner's attention on relevant elements in the 

segment; 

• Organizing and structuring the viewing to make the material 

memorable and relevant, not testing his/her memory; 

• Maintaining the integrity of the original segment. 

3. Follow-Up 

Help the learner understand the broader application of the material 
covered in the segment by: 

• Adding to or building on the layers of information presented in the 

video; 

• Extending the frame of usage of the material already learned; 

• Providing additional material to complete or supplement the 

portrait created by the video material. 
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Product and Process Aspects of NES/EFL Students' Persuasive 
Writing in English: Differences between Advanced and Basic 

Writers 

JIN-WAN KIM 

This paper was designed to investigate quantitative and qualitative 
differences of "product" and "process" variables between native and non- 
native "advanced" writers, and between non-native "advanced" and 
"basic" writers. It was found that a number of "product" and "process" 
variables that significantly differentiated between "advanced" and "basic" 
writers were also good predictors of student writing quality, and that 
these variables were distributed across all three types of aspects 
(linguistic, rhetorical, and strategic). Thus, it confirms that writing 
quality depended on all three aspects measured. This balanced approach, 
focused on "product" and "process" and on "quantitative" and 
"qualitative" method, provides an interactive and comprehensive view of 
L2 writing, and contributes to L2 writing instruction using realistic 
writing process strategies that take into account L2 writers' products 
resources. 

INTRODUCTION 

In an exploration of the unique nature of L2 writing, several empirical studies 
have recently focused on a direct comparison of ESL and NES writers and/or the 
LI and L2 writing of ESL students (Silva, 1993a). Studies looking at written 
texts ("product" variables) have outnumbered those dealing with composing 
processes ("process" variables) and, of these product studies, more have focused 
on rhetorical (discourse level) than on linguistic (sentence level and below) 
features (Silva, 1993a). In fact, no comprehensive studies that looked at both 
product and process variables at the same time were located. 

At the linguistic level (product), Ferris (1994) reported that native speakers had 
a greater number of clauses in their essays than did non-native speakers, that this 
length variable might be a good predictor of the holistic scores assigned to the 
essays, and that more advanced writers had higher means for "word length" and 
"words per clause" variables than did basic writers. It was also reported that non- 
native English writers used discourse markers more frequently at the sentence 
beginning than did native English writers, and that the frequent use of discourse 
markers might be another predictor of holistic essay scores. In the study of 
cohesion devices, Reid (1992) found that native English writers used far fewer 
pronouns and coordinate conjunctions, and far more subordinate conjunctions and 
prepositions than did non-native English speakers. 

At the rhetorical level (product), studies examining the quality of student 
writing showed that topical structure analysis is a good predictor of college 
students' writing quality (Witte, 1983; Connor & Farmer, 1990). A recent study 
of Ferris (1994) showed that native English writers had the lowest ratios of 
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subtopics to sentences (showing a greater degree of topical focus: i.e., good 
writers elaborating on a few arguments, rather than introducing many different 
subpoints), thus this subtopics per sentence variable became a good predictor of 
the writing scores. Rhetorical variables obtained from the analysis of Patthey- 
Chavez (1988) included the categorization of the "openings" (statement of the 
writer's opinion on the topic) and "closings" (writer's conclusions) as being 
personal, impersonal, mixed, or nonexistent. In this point, Ferris (1994) found 
that null closings differed between basic and advanced writers, with the more 
advanced writers more frequently providing closings (i.e., conclusions) than did 
the basic writers. Further, Wong (1988) showed both a qualitative and a 
quantitative difference in English and in Chinese, in the use of two types of 
rhetorical questions: 1) an "interrogative" and 2) an "assertive". Finally, Ferris 
(1994) used two categories for the study: reader inclusion and counterarguments. 
Results showed that the more advanced native writers most frequently used 
counterarguments, followed by the advanced non-native writers. 

Finally, at the strategic level, some studies on "process" variables have found 
that L2 writing was more difficult and less effective, and a closer look revealed 
some significant differences in the processes of planning, transcribing, and 
reviewing (Silva, 1993a,b/Forthcoming). With regard to planning, L2 writers did 
less planning at both the global and local levels (Campbell, 1990; Dennett, 
1985; Jones & Tetroe, 1987). With regard to transcribing, it was more 
laborious, less fluent, and less productive for ESL writers, reflecting a greater 
concern and difficulty with vocabulary (Hall, 1990; Krapels, 1990; Moragne e 
Silva, 1991). There was also evidence of differences with regard to revising: 
ESL writers reread their texts less often (Dennett, 1985; Silva, 1990) and 
reflected less on their written texts (Hall, 1990; Silva, 1990), than the NES 
writers. 

In a recent study by Hirose and Sasaki (1994), it was found that good EFL 
writers used writing strategies similar to those of good LI and L2 writers: 1) 
planning content 2) paying attention to content and overall organization while 
writing, and 3) revising at the discourse level. In particular, the good writers 
were especially concerned with content, whereas the poor writers gave no special 
attention to content. These findings provided evidence for the existence of a 
universal writing strategy regardless of the language used (Krapels, 1990; Zamel, 
1982). With regard to writing confidence, Hirose and Sasaki (1994) also reported 
that the good L2 writers had confidence in L2 writing because they had positive 
writing experiences. These researchers added that two factors (i.e., self-initiated 
writing experience and writing confidence) could be possible indicators of good 
L2 writers, although these two factors have not been identified as characteristics 
of good L2 writers in previous study. Crowhurst (1991) reported that knowledge 
of organizational structures of persuasive texts might contribute both to 
comprehension and to production of persuasive texts. This finding indicated that 
the use of discourse knowledge of persuasive/argumentative writing might 
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improve the quality of L2 writing. Finally, with regard to audience awareness, 
Connor (1990) suggested that student writers composing 
persuasive/argumentative essays would need to identify the presumed audience, 
recognize its potential opposition, and be consistent in their appeals. 

Because of the lack of studies that have included both "product" and "process" 
variables, there is a need for L2 writing research that is more evenly focused on 
"product" and "process" variables; on quantitative and qualitative methodological 
approaches; and on subjects of different levels of education, writing background 
or attitude, and writing process strategies. With this line, the present study 
attempted to identify the predictor (both product and process) variables 
significantly predicting the quality of English persuasive/ argumentative writing. 
Mainly, this study investigated quantitative and qualitative differences (in terms 
of linguistic, rhetorical, and strategic variables, and in terms of writing processes 
and background as well) between "advanced" and "basic" writers. The present 
study investigated the following research questions: 

1. Which variables, among all variables (product and process variables), 
significantly predict the quality of English persuasive writing? 

2. What are quantitative and qualitative differences between native and non- 
native "advanced" writers, and between non-native "advanced" and "basic" writers? 

METHOD 

Participants 

Participants in this study consisted of 1) NES students (28 American students, 
majoring in liberal arts) and 2) EFL students (28 Korean students in America, 
majoring in liberal arts, and 90 Korean students in Korea, mostly majoring in 
English). There were 79 undergraduate and 67 graduate students (44 men and 102 
women) among a total of 146 participants. Students of technology or natural 
sciences were excluded in this study because they are exposed to scientific 
discourse with a possibly different type of persuasive structure. 

Instruments and Procedures 

The instruments for student writers consisted of 1) persuasive writing tasks 
(Appendix 1), 2) follow-up retrospective questionnaires about the writing process 
(Appendix 2), and 3) writing background/attitude surveys (Appendix 3). For 
persuasive writing tasks, two structured prompts were chosen. These prompts 
had previously proven successful in L2 writing studies (Carson et al., 1990). 
The retrospective questionnaire and the writing background survey used in the 
study of Hirose and Sasaki (1994) were modified and adapted to the present study. 
The retrospective postwriting questionnaire included various questions about 
writing processes, writing strategies, the degree of attention to different 
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components while writing, the degree of attention to writing processes, the 
degree of audience awareness, and the use of discourse knowledge. The writing 
background/attitude survey included questions about the writer's basic 
background, educational background in regard to composition or rhetoric, 
instructional background on persuasive writing, prior writing experiences both in 
high school and at the university, self-initiated writing experiences, and writing 
confidence. 

For student writers, three tasks were conducted in the following sequence: 1) a 
persuasive writing task in English, 2) a follow-up survey on writing processes, 
3) a writing background and attitude survey. These tasks were assigned at one 
time. Randomly, half of them were asked to write about the topic prompt A, and 
the other half were assigned to write about the topic prompt B. For the tasks, the 
participants were not be informed in advance about the topic. They were not 
allowed to use any reference book, not even a dictionary. A limit of 45 minutes 
was imposed on the writing task. 

Design and Data Analysis 

First, this study evaluated linguistic, rhetorical, and strategic variables of the 
three groups’ writing products and processes, and used them as predictor variables 
for a multiple regression analysis. The dependent variable used in the multiple 
regression analysis was the score assigned by two English L2 writing 
specialists, who used the ESL Composition Profile (Jacobs, Zinkgraf, 
Wormuth, Hartfiel, & Hughey, 1981). Each writer's score was the average of the 
two raters' scores 1 and this score was taken as a measure of the overall quality of 
each piece of persuasive writing. 

As predictor variables (a total of 35), the following 10 linguistic variables, 13 
rhetorical variables, and 12 strategic variables were obtained from the tasks 
assigned to the students. Linguistic variables included "length" variables (number 
of words, number of clauses, word length, words per clause, and clauses per 
sentence), "cohesion" variables (pronouns, coordinate conjunctions, subordinate 
conjunctions, and prepositions), and "discourse marker." Rhetorical variables 
included "coherence" variables (parallel, sequential, and extended parallel 
progressions, subtopic per clause, and topical depth), "openings" (personal and 
impersonal), "closings" (personal and impersonal), "rhetorical questions” (type I 
and type II), "reader inclusion,” and "counterarguments." Strategic variables 
included "outline" (written outline), "revision," "composing process" (planning, 
transcribing, and revising), "writing confidence," "writing attention" 
(grammar/spelling, content, overall organization, and vocabulary choice), 
"discourse knowledge use," and "audience awareness." 

These predictor variables were quantified for the study, as follows. Length 
variables were counted and calculated for each writing through hand counts and 
through computer search functions (i.e., the "Tools" functions of Microsoft 
Word Version 6.0). Cohesion device variables (Reid, 1992) were also analyzed 
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and calculated through hand counts. Percent of pronouns was obtained from the 
total number of personal and demonstrative pronouns, divided by the total 
number of words in a single essay; percent of coordinate conjunctions was 
obtained from the total number of simple coordinate conjunctions, divided by the 
total number of words in a single essay. Percent of subordinate conjunctions was 
obtained from the total number of subordinate conjunctions, divided by the total 
number of sentences in a single essay; percent of prepositions was obtained from 
the total number of prepositions, divided by the total number of words in a 
single essay. The number of discourse markers was obtained from the total 
number of sentences that began with discourse markers (e.g., "First, however, on 
the other hand, consequently, in conclusion," etc.), divided by the total number 
of clauses in a single essay. 

Coherence variables were obtained through a topical structure analysis, 
developed by Lautamatti (1987) and followed by Connor (1990) and Ferris 
(1994). This analysis involved identifying the topical subject of each sentence 
and diagramming the relationships between sentences. These relationships were 
of three types; 1) parallel progressions (the topical subject was semantically 
identical to the topical subject of the previous sentence); 2) sequential 
progressions (the topical subject arose from the "comment" portion of the 
previous sentence); and 3) extended parallel progressions (the topical subject of a 
previous sentence was repeated following an intervening sequential progression). 
The values of these three types of progressions were obtained from the total 
number of relationships, divided by the total number of sentences in a single 
essay. Further, "subtopics per clause” was defined as the ratio of number of 
clauses to number of different subtopics in the writing, and "topical depth" was 
the longest string of sequential progressions in the writing. Openings (statement 
of the writer's opinion on the topic) and closings (writer's conclusions) described 
as being personal, impersonal, or null were obtained through an analysis based 
on Patthey-Chavez (1988). An example of a personal opening would be "I 
think..." or versus the impersonal "It seems..." Two types of rhetorical questions 
were counted: Type I, an "interrogative," the answer to which introduces new 
information; and Type II, an "assertive," a question to which a negative answer 
is implied (Wong, 1990). Finally, "reader inclusion" was calculated by counting 
the number of instances of use of first person plural pronouns (we/us) and second 
person pronouns (you), and "counterarguments" were obtained by counting the 
number of instances lexically tied to a hypothetical reader’s concerns (Hays, 
Brandt, & Chantry, 1988). 

Written outline and revision variables were identified from results of the 
retrospective questionnaire and writing samples. These variables were quantified 
as 0 or 1 according to absence or presence of written outlines and final revision 
after writing. Composing process variables (Silva, 1993a) were obtained from 
the results of the retrospective questionnaire, which used a 1 to 5 scale rating 
according to the degree of attention to each subprocess. Writing confidence 
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variable was obtained from the writing background survey and calculated by 
Writing Self-Efficacy Scales (Shell, Murphy, & Brunning, 1989). In quantifying 
writing attention variables, the 1-5 rating scales were used according to the 
degree of writers' attention to each component. Finally, discourse knowledge 
(Carrell & Connor, 1991; Wright & Rosenberg, 1993) and audience (Johns, 
1993) variables were obtained from the questionnaire. The use of discourse 
knowledge was quantified as 0 or 1 according to no use or use of that knowledge, 
and in quantifying audience awareness, the 0-2 rating scales were used according 
to the degree of awareness shown. 

In the present study, a multiple regression analysis was used to determine how 
well the variables predicted the quality of English persuasive writings. In 
addition, one-way ANOVA was used to explore quantitative differences between 
"advanced" and "basic" writers. In particular, this study used the qualitative 
analysis to investigate the role of such factors as writing background, writing 
processes, and writing attitude, in more detail. 

RESULTS AND DISCUSSIONS 

Relation of Product and Process Variables to Writing Quality 

Group means of writing scores were as follows: American students = 94.71, 
Korean students in America = 73.63, and Korean students in Korea = 65.71. 
From the one-way ANOVA, the group difference in English writing scores was 
very significant (F(2, 143) = 174.55, p < .00001). Further, Tukey-HSD test 
showed significant differences among three groups. 

Using SPSS, a stepwise multiple regression analysis was run with English 
writing score (total score) as the dependent variable and the 35 predictor variables 
as independent variables. Tables 1 and 2 show the results of this analysis: the 
stepwise multiple regression analysis and the ANOVA summary table for an 
overall multiple regression analysis. 

In the product aspect, three significant linguistic variables were one length 
variable (number of words) and two cohesion variables (percent prepositions and 
percent subordinate conjunctions), and one significant rhetorical variable was a 
closing variable (impersonal closings). In the process aspect, two significant 
strategic variables were revision and writing confidence. In this analysis, it was 
found that these six variables strongly influenced the quality of English 
persuasive writing (the proportion of score variance accounted for by the six 
predictor variables was 72%). 
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Table 1. Results of Stepwise Multiple Regression Analysis 



Step 


R2 


F(Eqn) 


sig F 


Variables 


Beta In 


1 


.50 


141.88 


.000 


In: Number of Words 


.71 


2 


.59 


101.50 


.000 


In: Writing Confidence 


.35 


3 


.64 


83.24 


.000 


In: Number of Clauses 


-.39 


4 


.67 


70.87 


.000 


In: Impersonal Closings 


.19 


5 


.69 


62.27 


.000 


In: Percent Prepositions 


.16 


6 


.71 


56.72 


.000 


In: Subordinate Conjunctions 


.18 


7 


.70 


67.64 


.000 


Out: Number of Clauses 




8 


.72 


59.42 


.000 


In: Revision 


.12 



Table 2. ANOVA Summary Table 


Source 


S 


df 


MS F sig F 


Multiple regression 
.0000 


18243.69 


6 


3040.61 59.42 


Residual 


7113.19 


139 


51.17 


Total 


25356.88 


145 





R^ = .72 (the proportion of score variance accounted for by the six variables) 



Differences between Advanced and Basic Writers 

In order to examine how the native and non-native "advanced" (NA and NNA) 
writers differed, and also how the non-native "advanced" and "basic" (NNA and 
NNB) writers differed, native "advanced” writers (22 students), non-native 
"advanced" writers (17 students) and non-native "basic" writers (17 students) were 
chosen from the partiipants. The selection was made on the basis of their 
English writing scores^. 

Quantitative Differences 

The results of one-way ANOVA’s showed many significant linguistic 
differences between native and non-native "advanced" writers (see Table 3). 
Among 10 variables, eight variables were significantly different between two 
advanced writer groups. Interestingly, it was found that there were only two 
significant linguistic differences between non-native "advanced" and "basic" 

O 

ERJC 

hfliflaffHEaaaa 



33 



26 Texas Papers in Foreign Language Education 

writers (see Table 4), both length variables (words per clause and number of 
words). 

Table 3. Linguistic Differences between NA and NNA Writers 



Mean 



Variable 


NA 


NNA 


F-ratio 


Number of Discourse Markers 


9.37 


21.74 


22.94* 


Number of Words 


607.86 


416.94 


14.75* 


Percent Pronouns 


6.54 


10.35 


14.00* 


Percent Prepositions 


0.01 


8.03 


13.95* 


Clauses per Sentence 


1.17 


1.07 


12.83* 


Number of Clauses 


35.27 


24.59 


10.62* 


Word Length 
Percent Subordinate 


4.81 


4.53 


9.95* 


Conjunctions 


28.63 


20.89 


5.21* 


*p <.05 








Table 4. Linguistic Differences between NNA and NNB Writers 








Mean 




Variable 


NNA 


NNB 


F-ratio 


-- Words per Clause 


7.80 


2.88 


19.59* 


Number of Words 


416.94 


294.94 


9.42* 



*p < .05 



As shown in Table 3, native ’’advanced" writers wrote longer essays (as 
measured by the length variables), used more effective cohesion devices (i.e., 
more subordinate conjunctions, more prepositions, and fewer pronouns), and less 
frequently used discourse markers than did non-native "advanced" writers. In 
addition. Table 4 shows that non-native "advanced" writers produced more words 
and more words per clause than did non-native "basic" writers. 

From the findings, it is confirmed that length is associated with the overall 
quality of writing (Crowhurst, 1991). Probably this is so because the longer an 
essay is, the more likely it is that the writer has done an adequate job of using 
significant variables of effective writing (Ferris, 1994). Further, it is supported 
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that limited use of personal pronouns in native speakers' essays marks more 
informational, detached, formal, and written discourse (Grabe, 1987; Biber, 
1985). Also, the greater use of prepositions and subordinate conjunctions in 
native speakers' essays indicated developmental writing maturity in English. In 
addition, it is confirmed that ESL writers tended to use more frequently the 
sentence beginning with a discourse marker than did native speakers who used a 
variety of ways to develop topical materials (Ferris, 1994). 

In the rhetorical aspect, one-way ANOVA's showed that native and non-native 
"advanced" writers differed in terms of counterarguments, impersonal closings, 
personal closings, rhetorical question type II, topical depth, and personal 
openings (Table 5). Table 6 shows that there was just one rhetorical difference 
(rhetorical question type I) between NNA and NNB writers. 

Table 5. Rhetorical Differences between NA and NNA Writers 



Mean 



Variable 


NA 


NNA 


F-ratio 


Counterarguments 


1.27 


.29 


14.22* 


Impersonal Closings 


.68 


.29 


6.42* 


Personal Closings 


.14 


.47 


5.81* 


Rhetorical Question Type II 


.77 


.00 


5.60* 


Topical Depth 


8.59 


6.94 


5.13* 


Personal Openings 


.27 


.59 


4.16* 



*p <.05 

As shown in Table 5, the differences in counterarguments and closings showed 
that native "advanced" writers more frequently produced two important elements 
of academic persuasive writing, anticipation of a reader's counterarguments and 
the need for a closing, or conclusion, to an argument, than did non-native 
"advanced" writers. This finding confirms Crowhurst's (1991) result that the 
more advanced writers more frequently produced these two elements in persuasive 
writing. Further, native "advanced" writers showed much deeper topical depth 
than non-native writers. It seemed that this topical depth was closely related to a 
greater degree of topical focus (i.e., good writers elaborating deeply on a few 
arguments, rather than introducing many different subpoints just at surface 
level). In addition, native "advanced” writers almost always used rhetorical 
question type n, whereas non-native "advanced" writers never used it. Further, as 
shown in Table 6, all of the non-native "basic" writers used rhetorical question 
type I, whereas non-native "advanced" writers nearly never used it. This finding 
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indicates that using rhetorical question type II was another writing strategy of 
advanced writers. 



Table 6. Rhetorical Differences between NNA and NNB Writers 



Variable 


NNA 


NNB 


Mean 

F-ratio 


Rhetorical Question Type I 


.18 


1.00 


4.80* 



*p <.05 



Finally, one-way ANOVA's showed the following strategic differences 
between native "advanced" and non-native "advanced" writers (Table 7). Further, 
non-native "advanced" and "basic" writers differed in terms of writing confidence 
and revision (Table 8). 



Table 7. Strategic Differences between NA and NNA Writers 



Mean 



Variable 


NA 


NNA 


F-ratio 


Writing Confidence 


95.72 


76.71 


34.28* 


Vocabulary Choice 


4.23 


3.29 


14.44* 


Written Outline 


.59 


.24 


5.36* 


Overall Organization 


4.32 


3.76 


4.56* 


Revision 


.68 


.35 


4.43* 



*p <.05 



These results confirm that writing confidence and revision were good predictors 
of the quality of writing, as already shown in Table 1. Further, native "advanced" 
writers used more written outlines, had more sensitivity of vocabulary choice, 
and paid more attention to overall organization while writing, than did non- 
native "advanced" writers. 




33 



Product and Process Aspects 



29 



Table 8. Strategic Differences between NNA and NNB Writers 







Mean 




Variable 


NNA 


NNB 


F-ratio 


Writing Confidence 


76.71 


65.19 


5.51* 


Revision 


.35 


.06 


4.88* 



*p <.05 



Qualitative Differences 

This qualitative analysis of the writing task and questionnaire showed that, 
before writing in English, a greater number of native "advanced” writers (59.1%) 
made their written outlines than did the non-native "advanced" writers (23.5%) 
and non-native "basic" writers (29.4%), and that the native "advanced" writers 
(68.2%) revised a little more than did the non-native "advanced" writers (35.3%), 
who revised much more than did the non-native "basic" writers (5.9%). It also 
showed that, whereas native "advanced" writers paid greater attention to their 
planning process, both non-native "advanced" and "basic" writers paid greater 
attention to their transcribing process. In terms of English writing confidence, 
native "advanced" writers (95.7% in the Writing Self-efficacy Scale) had more 
writing confidence than non-native "advanced" (76.7%) and non-native "basic" 
(65.2%) writers had. 

Self-ratings of how much attention they paid to grammar/spelling, content, 
organization, and vocabulary choice while writing in English indicated that 
native "advanced" writers paid a little more attention to each of these 
components, particularly to content, than non-native "advanced" and "basic" 
writers. The native "advanced" writers were a little more concerned with content 
(4.6 in the 5-point Likert scale) than were both non-native "advanced" and non- 
native "basic" writers (4.3). Interestingly, the non-native "advanced" writers paid 
more attention to grammar/spelling (3.3) than did non-native "basic" writers 
(2.8). Thus, it seemed that non-native "basic" writers were not very concerned 
about grammar/spelling. However, all of them had the same priority order of 
these writing components: content > organization > vocabulary > 

grammar/spelling. In terms of discourse knowledge use, the native "advanced" 
writers (77.3%) reported keeping a little more what they know about the typical 
way of writing a persuasive text in mind, than did the non-native "advanced" 
writers (64.7%), who used much more of their discourse knowledge for writing 
than non-native "basic" writers did (47.1%). Finally, in terms of audience 
awareness, native "advanced" (45.5%) and non-native "advanced" (47.1%) writers 
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tried more to write so that their specific readers could follow their argument 
easily, than did non-native "basic” writers (17.7%). 

The questionnaire asked whether the participants planned before beginning to 
write. Results showed that, before writing in English, a greater number of native 
"advanced" writers (77.3%) planned than that both of non-native "advanced" 
(52.9%) and of non-native "basic" (52.9%) writers. All of the planners, whether 
"advanced" or "basic" writers, concerned themselves with content before the 
actual writing. Half of the native and non-native "advanced" writers who planned 
thought about the direction, position, or organization of the composition, 
whereas some of the non-native "basic" writers thought about the meaning of a 
given topic itself. Further, the questionnaire asked how the participants managed 
to continue writing, and how much attention they paid to such items as 
grammar/spelling, content, overall organization, and vocabulary choice while 
writing. From the results, non-native "advanced" and "basic" writers seemed to 
approach English writing differently in terms of use of translation, fluency, and 
coherence. Over half of non-native "advanced" writers (64.7%) wrote directly in 
English without mentally translating, and only 17.6% of them used translation. 
Very few non-native "basic" writers (17.6%) wrote directly in English. Instead, 
58.8% of them initially generated their ideas in Korean and then translated them 
into English. Thus, it seemed that translation-free writing (i.e., direct writing) in 
L2 contributed to the quality of L2 writing. Over half of non-native "advanced" 
writers (58.8%) reported they wrote whatever came into their minds. Just a few 
of the non-native "basic" writers (35.3%) wrote whatever came into their minds, 
and most of them (94.1%) stopped and thought for a while, when they got stuck. 
Their low production rate (number of words per composition = 294.9) might 
have been due to their use of word-for-word or sentence-for-sentence translation, 
as well as frequent pauses while writing. Conversely, non-native "advanced" 
writers' translation-free writing and much greater fluency contributed to their 
fluent writing and high production rate (number of words per composition = 
416.9). In terms of writing coherence, 70.6% of non-native "advanced" writers 
tried to write so that the sentences would flow smoothly, whereas only 35.3% of 
non-native "basic" writers reported doing so. 

Almost all of the native "advanced" writers (95.5%) wrote in English with 
overall organization in mind, whereas just over half of non-native "advanced" and 
non-native "basic" writers (64.7%) did so. Also native "advanced" writers 
(81.8%) were more likely to focus on writing coherently than were non-native 
"advanced" writers (70.6%). Paying attention to the overall organization and 
focusing on writing coherence, therefore, were writing strategies that the 
advanced writers employed. Furthermore, a few of the native "advanced" writers 
specified their writing process as "Other." Some of these (13.6%) reported they 
had often referred to their initial outlines, and others (9.1%) reported they had 
kept a conversation in mind, as if they were speaking. Finally, the questionnaire 
asked what the participants did after writing. The analysis showed that over half 
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of native "advanced" writers revised (68.2%) the compositions once written, 
whereas fewer non-native "advanced" writers revised (35.3%). Almost none of the 
non-native "basic" writers reported revising (i.e., just 5.9% of them revised). 
Further, when revising, native and non-native "advanced" writers went about the 
task in different ways: the native "advanced" writers dealt with various levels of 
revision (e.g., form and content at the discourse level: spelling, sentence, 
sentence transition, and organization), whereas most of the non-native "advanced" 
and "basic" revisers only concentrated on form at the sentence level. 

From the writing background/attitude survey, the "advanced" (both NA and 
NNA) writers turned out to be older than the "basic" (NNB) writers. 
Interestingly, the "basic" writers consisted of a greater percentage of males 
(47.1%) than did the "advanced" writers (27.3% for NA or 29.4% for NNA). 
Also the "basic" writers consisted of a greater percentage of undergraduate 
students (82.4%) than did the native and non-native "advanced" writers (3 1 .8% 
for NA or 29.4% for NNA). In terms of English composition instruction, over 
half of native "advanced” writers (59.1%) had taken some English composition 
course, whereas only a few of non-native "advanced" (17.7%) and non-native 
"basic" (5.9%) writers had taken those courses. In addition, most of the native 
"advanced" writers (81.8%) had been taught how to write 
persuasive/argumentative essays in English, compared to 23.5% of the non- 
native "advanced" writers and none of non-native "basic" writers. Another 
difference was that non-native "advanced" writers had received more years of 
formal English instruction (11.9 years) than had non-native "basic" writers (9.7 
years). In addition, the non-native "advanced" writers had spent longer in an 
English speaking country (2.7 years) than had the non-native "basic" writers (0.7 
years). Both groups of writers, however, did not seem to differ very much in the 
writing score they had received when writing in LI. Further, the non-native 
"advanced" and "basic" writers were slightly different in terms of the amount of 
required English writing in high school. A few of the non-native "advanced" 
writers (17.6%) wrote more than five pages per semester, whereas none of the 
non-native "basic" writers did so. In particular, in terms of the amount of self- 
initiated English writing, almost all of the native "advanced" writers (90.9%) and 
almost half of the non-native "advanced" writers (47.1%) wrote more than 5 
pages per semester, whereas very few of the non-native "basic" writers (5.9%) did 
so. 

CONCLUSIONS AND IMPLICATIONS 

With the purpose of investigating quantitative and qualitative differences 
between native and non-native "advanced" writers, and between non-native 
"advanced" and "basic" writers, this study compared "product" and "process" 
variables involved in an analysis of the English persuasive writing of native 
English speakers and non-native English speakers. In this study, a number of 
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"product" and "process" variables that significantly differentiated between native 
and non-native "advanced” writers, and also between non-native "advanced" and 
"basic" writers were also good predictors of student writing quality. Interestingly, 
these variables were distributed across all three types of aspects (linguistic, 
rhetorical, and strategic), as shown earlier. Thus, it confirms that writing quality 
depended on all three aspects measured (linguistic, rhetorical, and strategic 
aspects). Though "product" variables seemed to be prominent in this analysis, it 
would be a mistake to privilege "product" (linguistic and rhetorical) variables at 
the expense of neglecting important "process" (strategic) variables. Conversely, 
it would be an equal mistake to ignore the "product" variables. In fact, many 
ESL/EFL writing teachers today accept that process writing pedagogies are 
applicable to academic writing, and are concerned with Final products as well as 
students' writing processes (Susser, 1994). Thus, this balanced approach provides 
the interactive and complementary effect of linguistic, rhetorical, and strategic 
variables on L2 writing. 

On this point, this study provides a useful model of L2 writing instruction 
using such features as [± linguistic], [± rhetorical], and [± strategic]. It is 
possible to categorize essays that have the nature of the advanced writers as [+ 
linguistic, + rhetorical, + strategic] and essays that fail to have that nature as [- 
linguistic, - rhetorical, - strategic]. In this way, student writers fall into one of 
various combinations of these features based on the relative strength of their 
linguistic, rhetorical, and strategic skills. If student essays are evaluated based on 
such a separate but balanced consideration of their "product" and "process" skills, 
they could receive a workable sequence of writing instruction that would help 
them improve their areas of weakness and lead them to mastery over writing in 
general. The qualitative analysis of this study showed that fluent writing, 
translation-free writing, focusing on writing coherence, and self-initiated writing 
were good strategies that the advanced L2 writers employed. This analysis was 
used in order to capture important information that was not captured by the 
quantitative analysis alone. Thus, this study was more evenly distributed in its 
use of quantitative and qualitative designs. 

The limitations of the present study suggest several directions for future 
research on L2 writing. Most of the Korean students in America in this study 
were international students, not ESL immigrant students. For more significant 
L2 writing research, studies examining writing products and processes of 
immigrant students should be added. Further, additional research should consider 
the effects of different types of writing tasks such as a comparison between 
"descriptive" or "expository" writing and "persuasive" or "argumentative” writing 
and the effects of different subject samples (various EFL students, not limited 
Korean students), in order to determine whether certain tasks are more 
problematic for non-native English writers and what sort of "product" and 
"process" variables are most effective in L2 (English) writing. Because this study 
dealt with just the first drafts of the students, rather than multiple-drafts, future 
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research should cover and the analysis of multiple drafts (Ferris, 1995), to 
explore the developmental process of learning to write in English. 

Finally, the present study has some implications for L2 writing pedagogy. 
Non-native English writers may come into L2 (English) writing situations with 
a different set of linguistic, rhetorical, and strategic expectations than do native 
English writers. Thus, certain elements of English persuasive writing such as 
cohesion devices, counterarguments, impersonal closings may be more 
problematic for non-native English writers than for NES writers, due to the non- 
native English writers relative lack of exposure to formal discourse in English. 
Accordingly, for non-native English writers, both modeling (especially self- 
initiated writing) and instruction in the writing structure of English persuasive 
texts may be effective in improving their L2 (English) writing quality 
(Crowhurst, 1991). In particular, as mentioned in the study of Silva (1993a), L2 
writing instruction should devote more time and attention across the board to 
linguistic, rhetorical, and strategic concerns. L2 writing instruction needs to 
include more work on planning and revising processes, so that L2 writers may 
make the actual writing more manageable, and more effective. Ideally, L2 
writing instruction needs to provide realistic writing "process" strategies for 
planning, transcribing, and revising that take into account L2 writers' "product" 
(linguistic and rhetorical) resources. 
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APPENDIX 1. Writing Prompts 



Writing Task (Prompt A) 
Time-45 minutes 



[Instruction] 

Write a persuasive/argumentative essay in English for the following topic. 
[Topic] 

Imagine that you are with a group of friends and the following topic comes up in 
discussion: "Some people believe that it is important for them to be part of a 





36 Texas Papers in Foreign Language Education 

group in order to achieve their goals. Others believe that they should try to be 
unique individuals in order to reach their objectives.” 

Which position do you prefer? In writing your composition, imagine trying to 
persuade your friends to take on your own position. Support your position with 
specific reasons and/or examples from your own experience, observations, or 
reading. 



Writing Task (Prompt B) 
Time-45 minutes 



[Instruction] 

Write a persuasive/argumentative essay in English for the following topic. 
[Topic] 

Imagine that you are with a group of friends and the following topic comes up in 
discussion: "As certain professions become more crowded, students of today are 
faced with a difficult choice. Some students choose a field they wish to enter 
even though job prospects are limited. Others choose a field in which they are 
less interested but which offers more job opportunities." 

Which position do you prefer? In writing your composition, imagine trying to 
persuade your friends to take on your own position. Support your position with 
specific reasons and/or examples from your own experience, observations, or 
reading. 



APPENDIX 2. Retrospective Postwriting Questionnaire for 
Writing in English 

1. Before writing 

1) Did you start writing right away? Yes No 

2) If your answer to 1) is "No," what did you do first? Write the kind of things 
you do specifically. (e.g„ I wrote down the outline. I thought about what I was 
going to write, etc.) 

2. In writing 

1) How did you manage to continue writing? Check as many as apply. If you 
did other than what is listed below, please specify. 

a. I generated ideas in LI first, then translated them into L2. (omitted for NES) 

b. I wrote directly in L2. (Omitted for NES) 

c. I wrote whatever came to my mind. 

d. I wrote with the overall organization in mind. 

e. I tried to write as much as possible. 

f. I tried to write so that the sentences would flow smoothly. 
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g. When I got stuck, I stopped and thought for a while. 

h. When I got stuck, I continued writing without stopping, 
and later went back to where I got stuck. 

i. Other (Please specify) 



2) When you were writing, how much attention did you pay to the following 
items? 



Not at all 



a. grammar 1 2 

b. spelling 1 2 

c. content 1 2 

d. overall organization 1 2 

e. vocabulary choice 1 2 



Little Fairly Very 

3 4 5 

3 4 5 

3 4 5 

3 4 5 

3 4 5 



3. After writing 

What did you do after writing? Please specify, (e.g., I did nothing once I finished 
writing. I reread to check whether the organization was appropriate or not and 
revised. I added to the first version where insufficient, etc.) 



4. In the whole writing process, how much attention did you 
following process? 



1) planning (before writing) 

2) transcribing (in writing) 

3) revising (after writing) 



Not at all Little Fairly 

12 3 4 

12 3 4 

12 3 4 



pay to the 

Very 

5 

5 

5 



5. Which of the following items did you do in your writing process. 

(Check only one) 

a. I developed my argument freely without considering my specific readers. 

b. I developed my argument with a slight consideration of my specific readers. 

c. I tried to write so that my specific readers can follow my argument easily. 

6. 1) In your writing process, did you keep what you know about the typical 
ways of writing an English persuasive/argumentative text in mind? 

Yes No 

2) If your answer is "Yes," list as many of the characteristics as you can think 
of that describes English persuasive/argumentative writing. 
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APPENDIX 3. L1/L2 Writing Background/Attitude Survey 

1. Age Sex 

2. Native Language 3. Major 

4. Educational Level (Undergraduate, MA, or Ph.D. students) 

5. About studying English (Omitted for NES): 

i) How long have you studied English at school? 

ii) Have you studied in an English speaking country? If so, where and how long? 

6. About taking composition or rhetoric courses: 

i) Have you taken any composition or rhetoric courses in an American 
university? 

ii) If yes, write the name of the course 

7. About the instruction of a persuasive/argumentative writing: Have you been 
taught how to write a persuasive essay in English? 

8. About writing in English 

i) Which of the following writing types have you done so far? 

(Check as many as apply) 

a. diary 

b. personal expressions on reading materials 

c. literary work (stories, poems, etc.) 

d. summaries or paraphrases of reading materials 

e. short papers or term papers 

f. research reports 

g. research proposal 

h. research articles 

i. other (Please specify) 

ii) Please estimate the amount of required writing you did while in high school. 

(Check only one) 

a. more than 20 pages per semester 

b. 11-20 pages per semester 

c. 5-10 pages per semester 

d. less than 5 pages per semester 

e. none 
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iii) Please estimate the amount of required writing you did while in the 

university. (Check only one) 

a. more than 20 pages per semester 

b. 11-20 pages per semester 

c. 5-10 pages per semester 

d. less than 5 pages per semester 

e. none 

iv) Which of the following kinds of writing have you done on your own (not 
connected to school work, that is, self-initiated writing)? 

a. diary 

b. personal expressions on reading materials 

c. literary work (stories, poems, etc.) 

d. summaries or paraphrases of reading materials 

e. short papers or term papers 

f. research repots 

g. research proposal 

h. research articles 

i. other (Please specify) 

v) Please estimate the amount of self-initiated writing you do. (Check only one) 

a. more than 20 pages per semester 

b. 11-20 pages per semester 

c. 5-10 pages per semester 

d. less than 5 pages per semester 

e. none 

9. About English writing confidence 

[On a scale from 0 (no chance) to 100 (completely certain), how confident are 
you that you can perform each of the following items? You may use any number 
between 0 and 100.] 

1 . Correctly spell all words in a one page passage 

2. Correctly punctuate a one page passage 

3. Correctly use parts of speech (i.e., nouns, verbs, adjectives, etc.) 

4. Write a simple sentence with proper punctuation and grammatical 

structure 

5. Correctly use plurals, verb tenses, prefixes, and suffixes 

6. Write compound and complex sentences with proper punctuation and 

grammatical structure 

7. Organize sentences into a paragraph so as to clearly express a topic 
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8. Write a paper with good overall organization (e.g., ideas in order, 

effective transition, etc.) 

9. Write something for personal purposes (e.g., writing a personal letter) 

10. Write something for academic purposes (e.g., writing a term paper) _ 



1 Interrater reliability was high:the Pearson correlation coefficient was .89. 

2 Native "advanced" writers had scores more than one standard deviation above 
the mean of the whole writing samples. Non-native "advanced" writers had 
scores more than one standard deviation above the mean of non-native speakers’ 
writing samples, whereas non-native "basic" writers were one standard deviation 
below that mean. 





An Investigation of the Effectiveness of Foreign Language 
Training Programs for TAs: An Exploratory Study 



ZENA T. MOORE and BARBARA BRESSLAU 

Training programs for Teaching Assistants have been in operation for 
over thirty years in many large state universities. Notwithstanding this 
fact, there is very little documented information on the effectiveness of 
the programs from the point of view of the TAs. The present study 
attempts to fill this gap by investigating what happens in foreign 
language department training programs at one large state university. 
Responses to a survey questionnaire by TAs revealed interdepartmental 
differences in effectiveness of training programs and suggestions as to 
their improvement. Specifically TAs believe that the programs should 
focus on practical pedagogical issues rather than on theoretical concerns 
and that the programs be ongoing rather than concentrated at the 
beginning of the semesters. Other needs include meaningful feedback on 
their teaching, skills for integrating material for classroom use, hands- 
on experience in audio-visual equipment, and more expertise on 
designing and scoring tests. 

A BRIEF HISTORY OF TASHIP 

TAship, as a university phenomenon, has been traced back to the late 19th 
century when Johns Hopkins University created what was to be the first graduate 
school in the country. Johns Hopkins* first TA was a special instructor in 
foreign languages employed to provide undergraduate instruction in Latin and 
Greek, (Rudolph, 1962). The early TAships at Johns Hopkins, as well as at 
other major universities, were created principally as a way of attracting students 
to graduate schools by providing financial incentives. Interestingly, many of 
these early beneficiaries were not expected to render any services in return. 

Half a century was to pass during which period TAs enjoyed this service-free 
financial support. By 1956, however, with the passing of the GI Bill, thousands 
of returning soldiers swelled the ranks of undergraduate enrollments in 
universities, especially in state universities. The increase in enrollment was so 
unprecedented that it was virtually impossible to hire assistant professors quickly 
enough to meet the expanding undergraduate population. This was particularly 
true in foreign language departments of large state institutions. 

As a consequence of the increasingly large enrollments at the undergraduate 
level a corresponding need for teaching staff mushroomed. A group of students 
whose eventual goal was to become part of the collegiate environment provided 
an expedient and quick solution. Many titles described members of this new body 
of students/teachers. They were assistant lecturers, assistant instructors, readers, 
proctors and teaching fellows among others. Regardless of the title, large 
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institutions wooed these graduate students to provide a much demanded 
undergraduate instruction, a demand that continues unabated even to the present 
date, for according to the results of a recent survey, (Huber, 1992), foreign 
language departments have steadily increased the number of their TAs to match 
the increasing enrollments. 

This nation-wide situation was described as a unique phenomenon of the 
modem research university, because the TA was simultaneously a teacher and a 
student. For a short while the efforts of TAs went uncriticised. Most universities 
operated on the assumption that TAs were just as qualified to teach as were 
certified teachers since both groups were holders of Bachelors' degrees with a 
major in a language. However, TAs generally qualified for employment based 
solely on their academic qualifications and enrollment in graduate school. That 
they had no training in pedagogy and did not have opportunities for pre-service 
and/or on the job training, as did student teachers with a Bachelors' degree, 
appeared to be insignificant. 

The seeming insignificance attached to pedagogical skills as an essential 
requirement for teaching continued through the sixties. According to findings in 
the ML A 1963 survey conducted among 52 foreign language departments, 60% 
of the departments surveyed did not offer any type of training for TAs 
(Hagiwara, 1976). Challenges to and criticism of the lack of importance placed 
on the pedagogic qualifications came from several quarters. 

The most vociferous complaints came from undergraduates across campuses 
about ineffective teaching assistants who lacked teaching skills, who could 
hardly speak English, and who were largely ignorant of the culture of the 
American schooling system. Large universities subsequently responded by 
instituting training programs designed to better equip the TAs with pedagogical 
skills, along with techniques and strategies for more effective teaching. The 
major challenge was to create programs that produced effective teachers in a 
minimum of time while maintaining the quality of instruction in lower level 
courses, (Hagiwara, 1976). 

TRAINING PROGRAMS IN FOREIGN LANGUAGE 
DEPARTMENTS 

For the following four decades program planners presented their course 
outlines and program details for public perusal, criticism and modeling For 
example, in 1957 Remak outlined the course for German TAs that was 
successfully implemented in his institute. Three years later, Winkelman (1960), 
described the Nebraska model also for German TAs. Natchman's 1963 program 
used in the French Department in an Illinois university followed, (Natchman, 
1963), and in 1967 Dalbor having taken what he called a realistic look at the 
training of Spanish TAs in Penn State presented his recommendations for 
designing more effective training programs, (Dalbor, 1967). There was a lull in 
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the literature for the next nine years. Then in 1976 Hagiwara broke the silence 
with the publication of the program which the French Department used for TA 
training in Michigan, (Hagiwara, 1976). In that same year, Azevedo, (1976) 
described the benefits and vigorousness of the pre-service orientation program 
used to prepare Spanish TAs in Urbana. 

Two major concerns guided training programs for TAs. One concern grew out 
of the widespread criticism of ineffective instruction, mentioned before. 
Undergraduates complained that their TAs were simply not good teachers. The 
second concern was for the TAs as students themselves (Azevedo, 1976, Remak 
1957). University professors were troubled by the likelihood that the incoming 
MA and Ph. D. students might enter the professoriate without any systematic 
preparation in pedagogy. They felt that the institutions had to respond to the 
needs of the graduate students as well. 

The following decade saw an increased awareness of the need to develop 
training programs of one sort or the other, notwithstanding some programmatic 
and administrative restrictions. Time was one of major structural restrictions. 
TAs generally came from other countries and could not be expected to be at the 
university long before the beginning of the semester without financial 
assistance. TA training, therefore, had to be conducted prior to the start of the 
semester, usually the week before classes commenced, and had to be ongoing for 
it to be effective. This was not always easy to achieve, for, although many 
universities recognized the need and importance of training programs for TAs, 
not many provided such training, (Hagiwara, 1976 ). 

Furthermore it was found that TAs performed a wide range of duties that 
included, providing feedback, giving advice, acting as counselor, functioning as a 
tutor, preparing and administering tests, grading students’ work, keeping 
students’ records and providing instructional media services. The over-demanding 
range of duties expected of the TAs evoked resistance and complaints on the part 
of TAs and attracted the attention of supporters who saw TAs as objects of abuse 
and as sources of ’’cheap labor” (Sammons, 1976; Sykes, 1988), for while 
enrollment figures grew, faculty size did not. According to Huber, (1995), 67% 
of institutions with 50% increase in enrollment figures increased their part-time 
faculty by only 31% compared to the 50% increase of TAs. To date a strong 
correlation still exists between increased enrollments at the undergraduate level 
and increased numbers of TAs used for instruction (Azevedo, 1990). 

Regardless of the reasons, be they to cut salary costs, or to provide real 
incentives through financial aid, the number of TAs continue to increase and TA 
training programs have become an institutionalized offering (Azevedo, 1990). 
During the 70s and the 80s heads of departments and TA supervisors evaluated 
their training programs and did indeed make recommendations and modifications 
(Schulz, 1980; Bennet, 1983; Di Donato, 1983; Azevedo, 1990; Magnan, 1993). 
Missing from the literature, however, is the lack of information on the 
effectiveness of these training programs from the TAs perspective. The work of 
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Ervin and Muyskens (1982), and more recently that of Gray and Buerkel- 
Rothfuss (1991) are among the few exceptions. 

THE STUDY 

The present study attempts to fill the gap in information about the 
effectiveness of training programs for TAs from the point of view of the TAs 
themselves. It is exploratory in nature since it focuses only on one large 
institute and on three main questions. The questions are: 

1 . What do TAs think of their training programs? 

2. What do they perceive to be effective teaching? 

3. Is there a fit between what is offered and what is needed? 

The University of Texas at Austin responded to the needs of its TAs by 
instituting training programs, commonly referred to as 398T courses, in the 
departments of French and Italian, Germanic Languages, Spanish and 
Portuguese. The 389T course are designed to: 1) introduce the teaching assistant 
to the most important teaching methodologies and 2) provide opportunities for 
gaining teaching experience under the close supervision of the course instructor. 

DATA COLLECTION AND ANALYSIS 

A survey questionnaire, presented in Appendix A, sought to elicit information 
in three categories: biographical information, responses to the training program 
and perceptions of the characteristics of an effective teacher. Most of the 
questions were intentionally open-ended to obtain insights into the topic. For 
purposes of this paper only data pertaining to the effectiveness of the training 
programs vis-a-vis the TAs' perceived needs will be discussed. Despite the 
shortcomings of the study, the information collected should give some insights 
to TAs' perceptions of their training programs. 

A total number of 110 surveys were distributed among the departments of 
French and Italian (24), Germanic Languages (36) and Spanish and Portuguese 
(50). A total of 56 surveys were returned constituting a response rate of 
approximately 51%. Data from the completed questionnaires formed the base for 
analysis. 

SURVEY FINDINGS 

A. Effectiveness of the Training Programs: Interdepartmental 
comparison 

Of the 56 informants who responded to the survey 33 did the training program. 
The remaining 23 indicated that they had taken a similar course at another 
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institution or had prior teaching experience, and thus were exempted from 
attending the training programs. While this number is admittedly small it may 
well be representative of the TA situation across foreign language departments. 

Table 1 shows that of the 33 respondents, 24 (72%) said that the 398T course 
prepared them to teach at UT. Approximately 98% felt that it did present various 
approaches to language teaching, and approximately 91% stated that it helped 
them adjust to the teaching approach adopted by their department. Eighty-four 
percent believed that the course allowed for discussion of problems related to the 
classroom. Approximately 91% thought that the course helped them to 
understand language teaching better, and 84% felt that the course helped them to 
understand their departments' pedagogic approaches better. 

Table 1. Responses of TAs in all Departments on Effectiveness of the 398T 
Course (In percentages. N=33) 



Summary of Questions 


YES 


NO 


Did the course prepare you for teaching at U.T.? 


72 


28 


Did it discuss various approaches to teaching? 


98 


02 


Did it allow for discussion of classroom problems? 


84 


16 


Did it provide for a better understanding of teaching in general?91 
Did it allow for an understanding of the approaches adopted 


09 


by the department? 


91 


09 



Interdepartmental Comparisons 

Table 2 reveals that in the Departments of French and Italian only ten (59%) 
of the 17 TAs took the 398T course, a very low figure compared to the 90% of 
TAs in the Department of Germanic Languages, and to the 75% in the 
Department of Spanish and Portuguese. While 90% mentioned that the training 
course helped them to understand language teaching better, only 60% thought 
that it helped them to understand the teaching mission of their department, 
indicating an unclear departmental mission and/or the lack of articulation 
between objectives of the course and classroom practices. The relatively low 
responses of 60% to the question of an understanding of various approaches to 
language teaching and 59% to preparedness to teach in the department suggested 
that there was significant need for improvement in the course. 
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Table 2. Responses from the French and Italian TAs on Effectiveness of the 
398T Course (In percentages N=17) 



Summary of Questions 


YES 


NO 


Did the course prepare you for teaching in the department? 


59 


41 


Did it discuss various approaches to teaching? 


60 


40 


Did it allow for discussion of classroom problems? 


90 


10 


Did it provide for a better understanding of teaching in general?90 
Did it allow for an understanding of the approaches adopted 


10 


by the department? 


60 


40 



German Department 

Table 3 shows consensus among the TAs in the German department. They all 
felt that the course discussed the various approaches to language teaching as well 
as the approach adopted by the department. They also all agreed that the course 
discussed classroom problems and that it gave them a better understanding of 
language teaching in general and of their department al approaches in particular 
and the majority of them, (90%), felt that it prepared them for teaching in the 
department. 

Table 3. Responses from German TAs on Effectiveness of the 398T Course 
(In percentage N=20) 



Summary of Questions 


YES 


NO 


Did the course prepare you for teaching in the department? 


90 


10 


Did you leant various approaches to teaching? 


100 


0 


Did it allow for discussion of classroom problems? 


100 


0 


Did it provide for a better understanding of teaching in general? 100 
Did it allow for an understanding of language teaching 


0 


in your department ? 


100 


0 



Spanish and Portuguese Departments 



Table 4 shows less consensus in responses of the Spanish and Portuguese 
departments. Seventy-seven percent believed that the 398T course prepared them 
for teaching in the department, and offered a better understanding of teaching in 
general. Eighty-five percent understood the departments’ approaches to teaching 
and 92% indicated that they learnt various approaches to teaching. On the less 
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positive side, 46% felt that the course did not allow for discussions of problems 
arising in the classroom. 

Table 4. Responses of TAs in Spanish and Portuguese Effectiveness of the 
398T Course (In percentages N=13) 



Summary of Questions 


YES 


NO 


Did the course prepare you for teaching in the department? 


77 


23 


Did you leam various approaches to teaching? 


92 


08 


Did it allow for discussion of classroom problems? 


54 


46 


Did it provide for a better understanding of teaching in general?77 
Did it allow for an understanding of language teaching 


23 


in your department ? 


85 


15 



Responses to Open-ended Question 

Responses to the open-ended question in this section revealed complaints 
about the overly theoretical nature of the course. Almost all TAs expressed 
strong feelings that actual classroom experience should be more emphasized and 
that the course was helpful only to beginners, to prospective instructors with no 
experience at all. Almost all TAs in the Spanish and Portuguese felt that there 
should be separate courses for Spanish and Portuguese TAs since their 
individual concerns were not being addressed. TAs also criticized the overly strict 
supervision. Another major criticism was leveled at the types of tests 
administered by the department. 

B. Characteristics of an Effective teacher 

Responses to this section of the questionnaire fell into categories dealing with 
professional competencies, personal competencies and perceived needs of the 
TAs. 

(a) Professional 

The majority of TAs felt that language competence and cultural competence 
were the two most important qualifications for effective teaching. They felt that 
knowledge of the literature, a background in linguistics and a background in 
applied linguistics were helpful but were not as important as linguistic and 
cultural skills. They also indicated that knowledge about theories of second 
language acquisition, or of language teaching/leaming were not essential for 
effective teaching. 
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The responses reflected similar concern about the overly theoretical nature of 
the training programs. On the one hand they expressed the need for hands-on 
experience and more practical experiences yet on the other they felt that active 
participation in workshops and training sessions were not their primary needs. 
Since most of the TAs were superior level speakers who had spent time in the 
target culture, it was not surprising that they would rate these skills in the way 
they did. At the same time, their responses to the effectiveness of the training 
programs appear to be contradictory to the responses to their professional 
competencies. 

(b) Personal 

Most respondents felt that maturity and a sense of responsibility were the two 
most important personal qualities of a good teacher. They also believed that 
teachers should have a commitment to teaching and be enthusiastic about their 
work in order to make language learning a positive and pleasant experience. 

C. Needs 

Most respondents stated that their greatest need was actual teaching practice. 
They felt that teacher training is a never-ending process, and although they 
believed that training did not necessarily make a good teacher, most of them 
believed in the importance of training, and more precisely in practical training. 
Again these responses were consistent with their criticisms of the departmental 
programs. 

The second most commonly expressed need was for feedback on their teaching. 
Once again, this expressed need seemed to be directly connected to the need for 
more practical experiences. Respondents made several suggestions by which they 
could be given feedback. The use of videotaping for observation and evaluation 
was the most frequently recommended strategy. Peer evaluation was another. 

Although the respondents did feel that they were qualified to teach, in that they 
felt they were proficient linguistically and culturally, they admitted that they 
needed pedagogic skills in at least four major areas of teaching. First, although 
they did have access to instructional supplementary material, they said that they 
lacked the skills necessary to effectively integrate the material into their lessons. 
Second, they said that they needed skills in teaching culture. Third, they felt that 
they would benefit if they had some insight into instructional psychology and 
fourth they needed training in conducting and grading tests in general and oral 
tests in particular. 

In the final category of the questionnaire which sought open comments in 
general, the most frequently cited complaint was overworked. The informants felt 
that teaching up to ten hours per semester and having large classes only made 
their teaching less effective. Fewer and smaller classes would provide them with 
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the opportunity to introduce a more communicative approach and could even be 
more effective than additional training. Another complaint was about the 
university’s policy regarding TAs. The third most frequently cited complaint was 
the low salary. TAs felt that they were performing a vital role and financial 
remuneration should reflect the university’s recognition of their important 
service. Finally, TAs expressed the need for counseling services that would allow 
them opportunities to discuss students' problems professionally and 
confidentially. 

SUMMARY AND DISCUSSION: THE FIT 

The responses to the survey indicated an overall awareness of the importance 
of teacher training. The majority of the informants agreed that the 398T courses 
do achieve the objectives by providing them with some training and in giving 
them the necessary theoretical background and tools to understand their 
assignment within their respective departments. At the same time, however, an 
impressive number of respondents wanted a more practical approach to teacher 
training, perhaps indicating that the short course undertaken before the classes 
commenced should be an orientation and an introduction, but that the training 
will be more effective and applicable if it is ongoing with direct connections 
between classroom practice and methodology. 

The TAs felt that there was not enough observation and especially not enough 
feedback on the observation. Several types of observation were suggested: 
observation of their own classes by supervisors and peers, observation of peers, 
watching videos of classes taught by other instructors, and watching videos of 
their own performance. Suggestions from fellow instructors and the feedback 
given by their students seemed to be more valuable than theoretical knowledge of 
approaches to teaching. This implies that more effective teacher training should 
turn more towards the classroom and utilize observations and discussions of 
videotaped classes as useful training devices. 

Some instructors also mentioned that the 398T courses might be too content 
specific, and that they would appreciate training that went beyond first semester 
courses and the institution. The concern with the quality of the instruction was 
further demonstrated in the solicitation of extra-materials, such as videos, 
pictures, realia to make instruction more interesting and meaningful. 
Nevertheless, they recognized that having instructional material was not enough. 
They needed to know how to use the material effectively and how to make 
instruction meaningful to the students. 

While the respondents acknowledged the importance of the theoretical 
component of the courses, they expressed the belief that they would benefit from 
more emphasis on the practical component. They did see themselves as involved 
in an ongoing process with the training course as a part of this process and not 
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the ultimate answer to it. In certain departments excessive workload and large 
numbers of students per class were linked to ineffective teaching . 

Overall, the results of the survey demonstrated that the informants believed 
that they had the academic qualifications to teach, that they were personally 
committed to their teaching, that they were aware of their responsibilities as 
teachers, knowledgeable about various approaches to language teaching, but they 
were not fully satisfied that the training programs were catering to their major 
needs. 

RECOMMENDATIONS AND CONCLUSION 

If the present trends continue, TAs will assume even greater responsibility for 
the task of preparing our language majors and consequently our future language 
teachers (Azevedo, 1990). Almost three decades ago. Freeman (1966) suggested 
that Foreign Language departments might want to consider creating special 
institutes for the preparation of their TAs if they hoped to improve the quality of 
foreign language. At that time the suggestion appeared too unrealistic. 

Given the fact that in over 70% of the nation's school districts teachers are 
presently charged with teaching students a foreign language, (Schrier 1994), 
foreign language departments must ask themselves whether they should be 
concerned with their influence on the development of these teachers. The findings 
from this exploratory study suggest that it is imperative that we give as much 
attention and prestige to pedagogy as we do to studies in literature, since the 
status has not changed much over the last two decades. Although there has been 
an increase in the level of academic qualifications of those directly involved with 
TA training and supervision (Schulz, 1983), they are still underpaid, overworked, 
non-tenure track instructors and/or lecturers, (Azevedo, 1990). The findings from 
the study show that our language majors, who most likely will be the future 
teachers, are being taught by TAs. If it is accepted that teachers teach as they 
were taught then a strong case must be made for more effective teaching at the 
university level by TAs. 

Prior to 1963 relatively little was done to train and supervise TAs, (Di 
Donato, 1983). To date there has been a slow but noticeable change, (Shulz, 
1979; Gibaldi and Mirollo, 1981). Universities, nevertheless, must continue 
their efforts to provide proper incentives for the supervisors and trainers of TAs. 
But more importantly, they have to develop training programs that are more 
effective. The traditional single course, like the 389T course in this study, 
accompanied by a one-week an orientation program is totally inadequate, since 
most of the time is spent on administrative rather than pedagogic concerns, 
(Magnan, 1993). An effective training program must include the following: 

i) A pre-service workshop to orient TAs 
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ii) An inservice program with a methods course either offered by the 

foreign language department or by the education department, 

iii) Regular classroom observation with feedback, 

iv) Practice teaching with feedback, 

v) Video-taped classes to provide for self evaluation and reflection, 

vi) Team- teaching, 

vii) Classroom observation of their peers 

viii) Training in designing, administering and scoring classroom tests. 

Finally, foreign language departments should employ only the best to teach 
lower level courses. The best will be linguistically competent, have superior 
knowledge of the culture, know how to integrate language and culture and know 
how to effectively use technology in instruction, (Rivers, 1983; Hancock 1981; 
Lafayette, 1983; Stein, 1966). Training programs for TAs must be redesigned to 
expose the future teachers to “teaching as a problem-solving skill”, not just to 
prepare them to deliver departmental courses. TAs as future language teachers 
must have opportunities to develop “decision-making skills” (Strasheim, 1984; 
Stem, 1983b), in addition to pedagogic skills. Training programs must produce 
teachers who are “reflective practitioners” and should shift towards a “more 
socially oriented knowledge base”, exposing them to a more critical perspective 
that would permit them to analyze and critique current teaching practices 
(Hudelson and Faltis 1993). Such training programs have the potential not only 
to produce highly motivated teachers but also to reduce attribution rates and to 
increase student enrollment, (Henderson, 1985). Universities should also make it 
mandatory that their training programs be evaluated externally and internally. All 
these changes call for greater collaboration between colleges of education and 
colleges of humanities. 

More detailed research on training programs offered in universities for the TAs 
is obviously needed but there should be no doubt about the necessity of 
instituting teacher preparation courses at the graduate level as recommended by 
Oksenholt in 1963 and as obtained in many universities. As Rivers argued, 
preparation should be considered “a necessity, not a luxury”, (1983b, 330). With 
the implementation of the National Standards (ACFTL 1995), we can expect that 
the need for TAs will probably be on the increase. The consequent demand will 
be to develop a more progressive approach to how we view pedagogy and an 
even more revolutionary way of how we define foreign language teacher 
education. 
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APPENDIX A 

This survey is meant to gather information about the Training Programs 
for Foreign Language TAs. 

Please take some time to answer the questionnaire. 

A . General Info 

1 . Please describe your status: 

2. What language do you teach: 

3 . Please describe your duties and responsibilities in your department. 

a. 

b. 

c. 

d. 

B. Teaching experience: 

1. How many semesters have you taught in your department? 

2. Which courses have you taught / are you teaching? 

3. How many times have you taught each course? 

4. Did you have any teaching experience prior to UT? YES NO 

If yes, where, for how long, and what type of courses did you teach? 
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1 . Have you taken the 398T course that your department offers? 

If yes, please check the appropriate answer 

- the course prepares the TAs for teaching in your department 

- the course discusses various approaches to language teaching. 

- the course discusses the teaching approach of the department 

- the course discusses problems that arise in the classroom 

- the course helps in a better understanding of language teaching 

-other, describe: 



YES 


NO 


YES 


NO 


YES 


NO 


YES 


NO 


YES 


NO 


YES 


NO 



2. If you answered NO to any of the above, please explain why. 



D. Characteristics of a Good Teacher 

1. What, in your opinion, are the characteristics of a good teacher. Please rank 
the items below, starting with 1 for the item you consider most important. 

(a) Competencies: Professional 
mastery of the language 

knowledge of the culture(s) in which the language is used 

knowledge of the literature(s) written in the language 

solid background in linguistics 

solid background in applied linguistics 

solid background in second language acquisition 

awareness of the current discussions on language teaching/leaming 

active participation in workshops, training sessions, conferences 

other, describe: 

(b) Personal 

self-confidence leadership objectivity 

kindness humor organization 

strictness authority ability to listen 

friendliness humility punctuality 

neatness confidence enthusiasm 

patience availability to students 

other, describe: 

E. Needs 

1. What, in your opinion, are the needs of a TA. Please rank the items below, 
starting with 1 for the item you consider most important. 

pre-service training 

in-service training 

possibilities for continuous language training 

offerings in methodology courses 

active assistance from coordinator 
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good teaching materials 

well designed courses / syllabi 

well equipped classrooms (tape recorder, VCR, etc.) 

financial assistance to attend training workshops and courses 

financial assistance to attend language courses abroad 

other, describe: 

F. Any other comments: 



Many Thanks for your Cooperation! 



The Concurrent Validity of Cloze Test with Essay Test among 
Korean Students 

SOYOUNG LEE 

This study investigated the concurrent validity of the cloze test with the 
essay test among Korean university students. A total of 129 students 
enrolled in freshmen English courses in Seoul, Korea, participated in 
this study. A seventh-word deleted cloze test and a descriptive essay test 
were administered to the students. A significant correlation between the 
two tests (r=.67 by the exact-word method and .64 by the acceptable- 
word method) was found. This result confirms the finding of two 
previous studies (Fotos, 1991; Hanania and Shikhani, 1986) that cloze 
tests can be an alternative to essay tests. This study has both theoretical 
and practical implications. Theoretically, the finding of this study 
seems to be an answer to the question of the construct of the cloze test. 
Practically, it suggests the use of the cloze procedure as a teaching 
device in classroom situations in Korea. 

INTRODUCTION 

Discrete-point tests have been widely used in evaluating students' English 
language proficiency in Korea. The advantages of discrete-point tests such as 
machine grading that enables to control massive number of examinees have been 
appealing. However, important concerns have been expressed in the second and 
foreign language literature that performance on a discrete-point test may indicate 
not overall language ability but intensive learning of isolated grammatical rules 
(Hanania and Shikhani, 1986). The same concerns have been also a critical issue 
in the English education field in Korea. The situation has been worrisome; six 
years of English learning during the period of middle to high schools results in 
only some reading ability but little ability of speaking, listening, and writing for 
most students. It is understandable considering that most English teachers in 
Korea are nonnative speakers and that, unlike listening and reading tests, it is 
difficult to develop objective scoring formats for speaking and writing tests. 
However, the disadvantages of a discrete-point test for the evaluation of the 
productive ability merit serious concern; such a test tells a great deal more about 
learners’ ability to recognize correct elements of the language than about their 
ability to use their knowledge effectively with a listener or a reader. 

Accordingly, the use of an integrative test for the evaluation of speaking and 
writing ability deserves attention. In particular, if there is an integrative test that 
is easy to prepare, administer, and score in evaluating the productive skills of 
speaking and writing, it will be very useful for English teachers in Korea who 
experience difficulties in dealing with the productive ability on tests. In this 
respect, cloze tests seem to be a promising approach. In fact, the usefulness of 
cloze tests has been recognized by some researchers. Hanania and Shikhani 
(1986) found significant correlations among a cloze, an essay, and a standard 
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ESL tests. Fotos (1991) reported that with careful preparation the use of cloze 
test as a writing proficiency measure can yield high validity and reliability 
estimates. 

This study has a purpose of confirming the concurrent validity of the cloze test 
with the essay test among Korean students. So far, many studies have attempted 
to validate cloze tests with various discrete-point tests (Bachman, 1985; Brown, 
1980, 1983) and shown a significant correlation between cloze tests and discrete- 
point tests. However, there has been little research on the validation of cloze 
tests with integrative tests to which cloze tests belong. Shohamy's (1983) study 
investigating the relationship between an oral interview and the cloze procedure, 
and Fotos’s (1991) and Hanania and Shikhani’s (1986) studies demonstrating 
high validity of cloze tests with compositions are a few studies. Thus, this study 
is to see whether cloze tests can measure the ability that essay tests measure 
among Korean students. 

METHODOLOGY 

Subjects 

The subjects in this study were 129 college first-year Korean students who 
were native speakers of Korean studying in two private universities in Seoul, 
Korea in March, 1995. Their majors were electronics, child and family, sport and 
leisure studies, Russian language and literature, chemical engineering, and 
management. Eighty-five students were male and forty-four students were female. 
They were enrolled in freshmen English classes that are required for all freshmen 
in the universities. 

The subjects in this study were a homogeneous group with regard to 
nationality, language background, age, and educational level. The students ranged 
in age from 18 to 20. While no measure of overall English language proficiency 
was available for the students, it can be said that all of the students had taken six 
years of formal English courses in middle and high schools prior to this study. 
Their 1995 college entrance English test scores indicated they were in the top 
10% group of those who took the test. 

Material 

This study needed two kinds of material: a cloze test and an essay test. The 
cloze test was prepared according to the procedures of Hanania and Shikhani 
(1986) and Fotos (1991). First, the students' reading ability level that would 
guide the choice of cloze texts was decided by Dale and Chall's (1948) readability 
formula using two college entrance English tests. Then, three different texts were 
picked from two textbooks designed for seventh or eighth graders in the U.S.A. 
Finally, three passages of about three hundred words for the rate of fifth word 
deletion were prepared to yield fifty blanks from the texts. From the passages, 
three different cloze tests were constructed. A pilot study led to revised cloze tests 
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with seventh deletion rate, thirty blanks, and more contextual information. As a 
pretest, the revised versions of the three cloze tests were tried out to 100 college 
first-year students. The form that showed the highest K-R 21 reliability estimate 
was selected for the final use, which is shown in Appendix A. 

The essay topic, “Plans for College Life,” was chosen since it was related to 
the students' general experience, motivated the students to write about, and was 
likely to elicit a variety of linguistic structures within the students’ previously 
learned lexical and syntactical knowledge. The topic did not require 
argumentation or definition of abstract concepts. The students were expected to 
write about one page on the given topic. 

Procedure 

The cloze and essay tests were administered to four English classes of 129 
students in March, 1995 with the cooperation of the English instructors. The 
cloze test was timed to take twenty minutes and the essay test was timed to take 
twenty-five minutes. The direction for taking the cloze test and scoring method 
were read aloud to the students as they read them silently. A sample test was 
used to orient the students to the testing format. The direction and sample test 
are shown in Appendix B. 

After the collection of all cloze tests, the essay test was administered. The 
instruction, topic, and scoring criteria of the essay test were also read aloud to 
the students as they read them silently. The students were given a blank page of 
paper to write the essay on. The instructions for taking the essay test and criteria 
by which the essays were graded are shown in Appendix C. 

Scoring of the tests 

The cloze tests were scored twice by both exact-word method, which counts 
only the same words as in the original passage as correct, and acceptable-word 
method, which counts any contextually appropriate words as correct as well as 
the words in the original passage. The grading of the cloze tests was done by the 
author and a native speaker. The author graded the cloze tests by the exact-word 
method and the native speaker did by the acceptable-word method. The criteria for 
an acceptable response were taken from the previous research (Stubbs and 
Tucker, 1974). 

The essay tests were graded independently by three native speakers. The graders 
used a nine-point Likert-type scale holistically with the consideration of three 
criteria. The nine-point scale and three criteria taken in grading the essays were 
from Reid and O’Brien (1981). The nine-point scale, going from 1 (poor) to 9 
(excellent), has five main levels: poor, fair, average, good, and excellent. The 
three criteria are: 1) English structure (the use of grammar and mechanics), 2) 
content (the quantity and quality of thought and diction), and 3) organization (the 
process of thought and support). 
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Prior to the actual evaluation of the essays, a standardization process was made 
to ensure the overall reliability of the holistic essay evaluation, which was a 
grader training session. The standardization process consisted of the steps taken 
in Reid and O'Brien (1981). First, the principles of holistic grading were 
explained and discussed by the author. Then, the description of the scoring 
criteria was distributed, read, and discussed. Finally, the graders were each given a 
group of three writing samples and asked to evaluate the samples in terms of the 
criteria. When each grader had finished, discussion of the scores followed in an 
attempt to arrive at a near unanimous rating (within one point). This practice 
session was repeated three times to refine the perceptions of the graders. 

Data analysis 

Statistical analysis was performed in three ways. First, descriptive statistics 
such as means, standard deviations, and reliability estimates of the two tests was 
calculated. For the cloze test, the K-R 20 method was used to determine internal 
consistency reliability. The interrater reliability for the essays rated by all three 
graders was calculated following the steps suggested in Perkins (1986). First, the 
evaluations of all raters were intercorrelated. An average of all three correlation 
coefficients was derived. To derive the average correlation coefficient, the Fisher 
Z transformation was used because the correlation coefficient was on a non-linear 
scale. Then, the average correlation coefficient was adjusted using the Spearman- 
Brown Prophecy formula to make the final reliability estimate reflect the number 
of graders. 

Second, the transformation of raw scores of the two tests was made for the 
purpose of easy comparisons; the maximum raw scores for the tests were 30 for 
the cloze test and 9 for the essay test. Raw cloze scores were recorded and 
multiplied by 3.3 to obtain percent scores for total cloze performance. Raw essay 
scores given by the three graders were averaged and then the average scores were 
multiplied by 11.1 to obtain percent scores for total essay performance. Finally, 
the Pearson’s product-moment correlation was performed to see how the two 
tests correlated with each other. It indicated the degree of commonality among 
the tests. 

RESULTS 

The findings on basic information of the two tests, such as means, standard 
deviations, and reliability estimates are first presented. Then, the result of the 
correlational analysis is presented. 

Table 1 presents the mean, standard deviation, and K-R 20 reliability estimate of 
the raw cloze test scores. 
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Table 1. 

Mean, standard deviation, and reliability of the raw cloze scores (n=129) 



Scoring Method 


Mean 


SD 


Reliability 


Exact-word 


10.21 


3.88 


.73 


Acceptable word 


13.42 


4.75 


.78 



* The maximum cloze score was 30. 



Table 1 shows that the cloze mean score of the present study by the exact-word 
scoring method was relatively high in comparison with the cloze mean scores 
obtained in previous research. In Fotos’s (1991) study, the mean score of was 12 
out of total score of 50. In Brown’s (1993) research, the means of fifty cloze 
tests ranged from 1.02 to 9.92 out of total score of 30, though his “natural” 
cloze tests were found to be more difficult than typical cloze tests. 

The mean score by the acceptable-word scoring method was higher than the 
mean score by the exact-word scoring method. The difference of the mean scores 
between the two methods was 3.21. In other words, the mean score by the 
acceptable-answer scoring method was about 10% higher than the mean score by 
the exact-answer scoring method. 

The cloze test reliability was estimated using the K-R 20 method. The K-R 21 
method used in previous research (Fotos, 1991; Hanania and Shikhani, 1986) 
was criticized for its underestimate or sometimes overestimate of reliability in 
spite of the advantage of easier calculation (Brown, 1993) and thus avoided in the 
data analysis of the main cloze test; the assumption of the K-R 21 procedure that 
all of the items are of equal difficulty is rarely met on real tests. The reliability 
estimates were fairly high, which means the cloze test is reasonably reliable to 
produce consistent results. 

The correlation coefficient between the exact-word and acceptable- word scoring 
methods was .94, which was significant (p < .01). The coefficient of 
determination, the value obtained by squaring the correlation coefficient, 
indicates the percentage of overlapping variance between the two scoring 
methods. The .88 coefficient of determination indicates the two methods were 
strongly related to each other. 

Table 2 presents the mean, standard deviation, and interrater reliability estimate 
of the raw essay test scores. Table 2 shows that students’ writing proficiency 
was slightly below the average, which was 5 on the nine-point scale going from 
1 (poor) to 9 (excellent). As the small value of standard deviation indicated, most 
essays (about 60% of the total essays) were in the category of 4 to 6, which were 
below average, average, and above average. The high .89 interrater reliability 
estimate of the essay tests calculated using the average of intercorrelation 

ERIC 65 

hfliflaffHHaaaa 



62 Texas Papers in Foreign Language Education 



coefficients and the Spearman-Brown Prophecy formula confirmed that holistic 
evaluations of essay tests are reliable (Flahive and Snow, 1980) as well as valid. 

Table 2. 

Mean, standard deviation, and reliability of the raw essay scores (n=129) 



Mean 


SD 


Reliability 


4.22 


1.23 


.89 



* The maximum score was 9. 



Finally, in Table 3 are listed transformed scores of the two tests from raw to 
percent correct in order to make easier comparisons among the two tests. 

Table 3. 

Means and standard deviations of the percent correct scores of the cloze and essay 

tests (n= 129) 



Test 


Mean 


Standard Deviation 


Cloze (Exact- word) 


33.69 


12.81 


Cloze (Acceptable word) 


44.28 


15.68 


Essay 


46.87 


13.62 



The maximum score of each test was 100. 



The relationships between the essay and cloze scores 
Table 5 presents the relationship between the cloze scores and the essay scores, 
which was expressed by the Pearson’s product-moment correlation coefficients. 

Table 5. 

The relationships among the cloze and essay (n=105) 



Paairs of Tests 


r 


Essay and Cloze (Exact-word) 


.67 


Essay and Cloze (Acceptable word) 


.64 



Both correlations were significant at pc. 01 
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The correlations indicate a degree of commonality between the tests and 
confirm their validity as tests of language proficiency. The .67 correlation 
coefficient between the essay and cloze scores is noticeable in that lower 
correlation coefficients between essay and cloze scores were found in previous 
studies: .51 from Hanania and Shikhani’s (1986) samples of college applicants 
and .46 from Fotos’s (1991) samples of college freshmen. Besides, considering 
that the students in this study were restricted samples in terms of their college 
entrance English test scores, the high correlation coefficient was remarkable; 
lower correlation coefficients are expected for a truncated sample. 

DISCUSSION 

This section discusses the results of the data analyses in light of possible 
explanations coming from the existing literature. 

Descriptive statistics 

The cloze mean score of the present study is not in line with the mean scores 
shown in previous studies (Chavez-Oller et al., 1985; Fotos, 1991). Fotos s 
(1991) conclusion that “cloze test mean scores are rather low regardless of the 
institution, the language tested, or the type of students, especially when exact 
scoring is used” (p. 329) does not fit in the present research. The higher cloze 
mean score of the present study appears to result from the analysis of readability 
level of the participants of this study prior to the selection of the possible cloze 
passages. Unlike the previous research, this study used Dale and Chall's (1948) 
readability formula to decide the students' reading level correctly. The evaluation 
of the students’ reading ability was found to be the readability formula score of 
6.64, indicating the reading level of seventh to eighth graders in the U.S.A. This 
evaluation on the basis of a scientific calculation led to a more correct selection 
of cloze passages than that based on testers' subjective judgments or random 
choices made in the previous studies. 

Besides, the revision of the cloze passages made after a pilot study and the 
final selection of the cloze test made after a pretest seem to contribute to the 
higher cloze mean score. The importance of careful preparation and selection of 
cloze tests emphasized in Fotos’s (1991) study is again confirmed. This 
confirmation is in line with a study by Brown (1993) in which the conclusion 
was drawn that “tester/researcher intervention is necessary for developing sound 
cloze test” (p. 1 10). In fact, a tester or a researcher’s cloze selection decision on 
the basis of the most desirable descriptive characteristics such as reliability 
statistics makes cloze tests reliable and valid tests of overall second or foreign 
language proficiency. However, since the variation in the effectiveness of cloze 
tests has been found to be considerable (Brown, 1993), the view of cloze tests as 
an automatic indicator of second or foreign language proficiency is not advocated 
in this study. 
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The .73 (by the exact-word method) and .78 (by the acceptable-word method) 
reliability estimates of the cloze test in this study are noticeable considering that 
homogeneous samples in ability levels are often associated with low reliability 
estimates (Brown, 1993). Despite the restricted range of the cloze scores found in 
the homogeneous group of the subjects in this study, the fairly high reliability 
estimates are reported indicating the cloze test is reasonably sound. 

The strong relationship between the two scoring methods indicated by the 
correlation coefficient confirms the findings of the previous research (Oiler, 
1973; Stubbs and Tucker, 1974) and suggests the easier exact-word scoring 
method could be appropriate for use by nonnative speaker teachers. Another 
support for the use of the exact-word scoring method, which has been criticized 
for discouragingly yielding low scores to the cloze test takers, comes from the 
about 10% difference of the mean scores between the two scoring methods found 
in this study. The difference of the mean is quite small in comparison with 
Brown’s (1980) study in which the mean by the acceptable-answer scoring 
method was 71% higher than the mean by the exact-answer scoring method. 

The near-average mean score and the small value of the standard deviation for 
the essay test seem to show that the subjects in this study are homogeneous 
with regard to English writing proficiency. It should, however, be noted that just 
one sample of essay may be insufficient to evaluate the writer’s composition 
ability (Henning, 1987). In fact, after finishing the essay scoring, one of the 
essay graders mentioned that nearly all essays were similar in terms of the 
amount of writing and writing style as well as the content of writing. Thus, 
rather than single compositions often criticized for unreliable representations of 
writing proficiency, several writing samples whose topics are various are 
recommended as a more correct measure of writing (Henning, 1987). 

The .89 interrater reliability estimate for the essay scores in this study 
confirms holistic evaluations can be reliable. The subjectivity involved in 
holistic scoring due to bias, fatigue, and shifting standards from one paper to the 
next is reduced by several techniques: establishment of criteria to focus graders’ 
attention on significant aspects of the writing, training of graders until they 
achieve close agreement in the assessment of the same paper, and three 
independent ratings of each essay. 

Validity of the cloze test as a writing measure 

The observed high correlation between the essay and cloze tests reflects the 
concurrent validity of the cloze test as a writing measure. The common 
integrative nature between the essay and cloze tests is proved. This means both 
tests require students to draw upon several language skills, which are production 
of language rather than recognition of correct items. Hanania and Shikhani 
(1986) and Fotos (1991) viewed the commonality between the two integrative 
tests is related to the testing of higher-order language abilities. 
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Limitations 

There are some limitations in this study. First, in spite of the efforts put in 
the design of the cloze test used in this study, the cloze test has not resolved the 
problems of generalizability and the equivalence of items across different forms. 
This situation is related to the validity issue. Considering the remark that “... 
validity is related more to the individual items of the test than to general 
variables, such as rate of systematic deletion ...” (p. 108, Hanania and Shikhani), 
the cloze test needs the identification of item characteristics for a more suitable 
test format. 

Second, some study (Stansfield and Hansen, 1983) shows that students’ field 
independence has a consistent relationship with cloze performance. The 
correlation found between field independence and performance on cloze tests 
seems to indicate a cognitive style bias operates in cloze solutions. On the other 
hand, test anxiety was found to affect students’ cloze performance (Oh, 1992; 
Shohamy, 1982). Considering that affective factors have a strong impact on 
students’ performance in testing and learning, the examination of the effect of 
test anxiety on cloze performance should be needed in research on cloze tests. 
Thus, certain nonlinguistic characteristics, which have not been considered in 
this study, are worthy of further investigation and interpretation. 

Finally, the evaluation of a student’ writing proficiency through an essay 
sample has been controversial, especially, when holistic scoring is used. Single 
compositions are said to be unreliable representations of writing proficiency 
(Arthur, 1979). Thus, to reduce the threats to reliability, the elicitation of 
multiple writing samples is recommended to control for the fact that writing 
proficiency may vary with topic (Henning, 1987). With more reliable 
evaluations of students’ writing proficiency, the relationship between cloze and 
essay scores would be accurately revealed. 

Implications 

The result of this study has both theoretical and practical implications. 
Theoretically speaking, the results of this study may be an answer to the 
question of what underlying factors are involved in cloze test performance. The 
strong relationship between the cloze and essay tests confirms writing 
proficiency may be an important factor to explain cloze performance. Further 
investigation of the factors to influence cloze performance, both linguistic and 
nonlinguistic, may lend insights into how such factors operate in cloze 
solutions. 

Practically speaking, the confirmation of the cloze procedure as a valid 
language proficiency test suggests the use of the procedure beyond a testing 
format. It can be used as an effective teaching device. Hinofotis (1987) suggests 
some variations in the cloze format suited for classroom use such as use of the 
cloze format in an exercise in listening comprehension. Such a modified use of 
the cloze procedure is found in other research (Buck, 1988) and indicates 
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individual teachers can vary the cloze format to suit their instructional needs in 
the classroom. 

Conclusion 

This study aims to determine whether cloze tests can measure Korean students’ 
English writing proficiency. Result supports the view that cloze tests can be 
used as an integrative measure of writing proficiency. However, for the effective 
and accurate use of cloze tests, careful preparation of cloze tests should be done. 
In addition, nonlinguistic factors such as cognitive and affective style bias should 
be considered in the interpretation of cloze test scores. 
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APPENDIX A 

Cloze test 

As the head chef of a 120-seat formal Northern Italian restaurant, I have 
cooked many beautiful and delicious meals. The most important critics, our 
customers, have for four years in a row named us their favorite restaurant in this 
city. All of the hard work has paid off. 

This career of mine was not (1) with any foresight at all. 

Rather, (2) kind of snuck up on me (3) blossomed 

unexpectedly. 

The beginnings of my (4) were quite typical: I was out 

(5) college, broke, and in debt. When (6) set out to look 

for a (7) , my criteria were simply to find (8) that I 

enjoyed, and to use (9) natural ability. I decided to try 

GO) • 
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At first, I wondered whether I (11) made a wise choice. 

My first (12) position exposed me to all of (13) evils 

of the restaurant business. There (14) the owners who cared only 

about (15) bottom line, and the managers who (16) 

nothing about food and who cared (17) less. 

I wanted a job where (18) cooking was taking place. I 

looked (19) a place where quality products would (20) 

handled professionally, and where I would (21) able to contribute 

what I had (22) offer. 

So, eight years ago, I (23) an apprentice here at 

Giovanni’s. Working (24) this kitchen proved to be a 

(25) “education” for me. I gradually became (26) of how 

temperature, cooking times, and (27) affected the look, texture, and 

flavor (28) the food I was preparing. 

On (29) down side, the hours were murder. 

(30) kitchen was as hot as a furnace. I worked under a disagreeable 

chef. 

This man eventually left, and that was the turning point for me. 
Although I had very little experience, almost no trained help, and a hot summer 
ahead, I took over as the new chef. 

Answers for cloze test 

1) planned 2) it 3) and 4) career 5) of 6)1 7) job 8) something 9) my 
10) cooking 11) had 12) cooking 13) the 14) were 15) the 16) knew 17) 
even 18) real 19) for 20) be 21) be 22) to 23) became 24) in 25) 
good 26) aware 27) ingredients 28) of 29) the 30) The 



APPENDIX B 

Directions for Taking Cloze Tests 

At the bottom of this page is a sample of a new kind of test. The test is 
made by copying a few paragraphs from a book. Every seventh word was left out 
of the paragraphs, and thirty blank spaces were put where the words were taken 
out. 

Your job will be to guess what word was left out of each space and to 
write that word in that space. It will help you in taking the test if you remember 
these things: 

1 . Write only one word in each blank. 

2. Try to fill every blank. Don't be afraid to guess. 

3. You may skip hard blanks and come back to them when you have finished. 
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4. Wrong spelling will not count against you if we can tell what word you 
meant. 

5. All blanks can be answered with ordinary words. 

Sample Test 

Below is a sample of the test to help you to be familiar to the main 
test. Fill each blank with the word you think was taken out. You may check 
your paper when you finish it by looking at the answers which are written at the 
bottom of the page. 

What is a friend? More than (1) father, more than a brother: A 

(2) companion. With him, you can conquer (3) impossible. 

1) a 2) traveling 3) the 

APPENDIX C 

Instruction 

This is a test of your ability to write an essay in English. When I give 
you the test paper, you will have twenty five minutes to write an about one page 
essay. The topic you are write about is stated on the test. You may take a few 
minutes, if you want, at the beginning of the test, to make notes or an outline 
for your essay on the back of the paper you receive. If you do not have enough 
space on the front of the page, you may continue on the back. 

Criteria of evaluation 

Your essay will be evaluated as to your ability to communicate effectively in 
writing. We will look at three qualities in particular: English structure, content, 
and organization. 
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Trends of Language Learning Strategy Research 
MI-AE LEE 

Almost all language learners initiate some action to overcome their problems 
in language learning and seek the best and quickest way in which they can 
improve their foreign language skills. These techniques, steps, actions, or 
behaviors taken by language learners to enhance acquisition are called language 
learning strategies (Oxford and Cohen, 1992; MacIntyre, 1994; Chamot and 
Kupper, 1989). During the last two decades or so, research on language learning 
strategies (LLS) has boomed, and succeeded in opening the eyes of researchers to 
the possibilities of the learner exerting control over the learning process. 
Nevertheless, the literature may cause confusion because theoretical assumptions 
and claims often vary from researcher to researcher. The author, therefore, feels a 
strong need to summarize the trends of LLS studies for better understanding of 
the literature. 

The present paper attempts to chronologically examine previous studies on 
LLS and then group the individual studies into the following sets: 

1. 1970s Good language learner studies 

2. 1980s Cognitive psychology-based research 

3. 1990s Vygotskian approach 

A. Style-strategy linkage 

B. Sociocultural theory-based research. 

Interestingly enough, a common psychological basis can be found underlying 
studies within each set. This tendency shows that language learning strategy 
research basically conforms to a psychological approach to understanding human 
learning. 

The first set, carried out mainly in the 1970s, includes such researchers as 
Wong-Fillmore (1979), Rubin (1975), Stem (1975), and Naiman et al. (1975), 
who focused on characteristics and strategies of the good language learner. 
Furthermore, such researchers suggested that language teachers teach poor 
language learners the strategies used by good students. The second school of 
language strategy experts, those of the 1980s, enriched the previous research 
results. They are represented by O’Malley et al. (1985a; 1985b; 1990; etc.). 
They sought an overall accounting of cognitive language learning. 

The 1990s have witnessed a divergence between two slightly different research 
groups. Both groups reflect Vygotskian theory of educational psychology in that 
they emphasize influences of culture and social interactions on human learning. 
However, these groups still find differences in their research methodology and 
claims. Therefore, the present paper prefers to consider those groups as two 
different strands of research in 1990s. The former school of style-strategy linkage 
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has been led by Oxford and her colleagues ( 1989; 1992; 1993; 1995; etc.), while 
the school based on sociocultural theory includes Lantolf and Appel (1994), 
Wertsch and Tulviste (1992), and Donato and McCormick (1994). The paper will 
briefly review those four sets. 

1. 1970s - Good language learner studies 
Researchers in the 1970s attempted to define the strategies that successful 
language learners employ, and then they tried to draw a profile of good language 
learner behaviors. Reflecting the shift from behaviorism to cognitivism in 
psychology, researchers paid attention to learners and tried to see what learners do 
to assist in their own learning. 

A. Wong-Fillmore’s study (1979) 

After her study with Spanish speaking children, Wong-Fillmore discounted 
aptitude as a basis for learning variation that until then had been widely regarded 
as one of the crucial individual difference factors in language learning. Instead, 
she emphasized the cognitive and social strategies (shown by Table 1 below) 
employed by the children that helped in their language learning (Wong-Fillmore, 
1979; Skehan, 1989). Agreeing with Grice’s cooperative principle in 
conversation, which is that the speakers assume that the listeners can understand 
them so that communication between them is possible (Grice, 1975), Wong- 
Fillmore further proposed that some social strategies play a more important role 
in language acquisition than do cognitive strategies. Wong-Fillmore stated: 

The crucial factor was this: Because the friends believed that the learners 
could understand them, and that communication between them was 
possible, they included them in activities and conversations, and this 
allowed the learners to assume roles in social situations and activities 
that made sense to them, and gave them an opportunity to observe and 
acquire the kind of language children use in these activities, despite their 
initial inability to speak or understand the language. 

(Wong-Fillmore, 1979: 209-10) 

Wong-Fillmore also implied that children who focus on details are apt to have 
difficulty in assembling these details in a larger system. 

One may argue from this study that the result would reflect very much the 
quality of the subjects and their language learning context, which means that 
children are likely to be more interested in establishing social relationships than 
in learning a language. 
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Table 1. Classifications of Language Learning Strategies by 



Researchers in the 1970s 




Author Primary strategy 

secondary strategies 


Representative classification 


Rubin ( 1 98 1 ) Direct strategies 


Clarification/verification 

Monitoring 

Memorization 

Guessing/inductive inferencing 
Deductive reasoning 
Practice 


Indirect strategies 


Creates opportunities for practice 
Production tricks 



Naiman et al. Active task approach Responds positively to learning 

opportunity or (1978) seeks and 



Realization of 


exploits learning environments 
Adds related language learning 
activities to regular classroom 
program 
Practices 

Analyzes individual problems 



language as a system Makes L 1/L2 comparisons 

Analyzes target language to make 



Monitoring L2 
performance 


inferences 

Makes use of fact that language is a 
system a means of communication 
and interaction 

Emphasizes fluency over accuracy 
Seeks communicative situations 
withL2 speakers 
Management of affective demands 
Finds sociocultural meanings 
Copes with affective demands in 
learning 

Constantly revises L2 system by 
testing inferences and asking L2 
native speakers for feedback 
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Wong-Fillmore Cognitive strategies Assumes relevance of what is 

being said (1979) 

Gets some expressions 
Looks for recurring parts 
in formulae 

Makes the most of what 
he/she has acquired 
Works on the big things 
first 

Social strategies Joins a group 

Gives the impression 
he/she can speak the 
language 

Counts on his/her friends 



Source: O’Malley and Chamot (1990: 4-5) and Skehan (1989: 81) revised 

B. Rubin (1975), Stern (1975), and Naiman et al.’s (1975) 
profiles of good language learners. 

Based on observations and interviews. Stem and Rubin separately presented 
influential lists of characteristics of successful language learners (Rubin, 1975; 
Stem, 1975). Influenced by Stern’s idea about the characteristics of good 
language learners, Naiman, Frohlich, and Todesco also interviewed a sample of 
effective and less effective language learners in order to explore such traits. The 
present paper will show the profile by Stem and the one by Naiman, Frohlich, 
and Todesco, as examples. 

Stem’s Profile 

Stem (1975: 311-316) compared and contrasted his profile of good language 
learners with that of Rubin. He suggested ten strategies that differentiate the 
good learner from the poor one. These strategies include the following: 

A personal learning style or positive learning strategies. Good 
learners will match their own styles or strategies with their own 
learning. 

An active approach to the learning task. Compared to poor 
learners, good learners are active, and they do not depend 
heavily on the teacher. 

A tolerant and outgoing approach to the target language and 
empathy with its speakers. This feature is consistent with Rubin’s 
observation that “the good language learner is not inhibited”. 
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d. Technical know-how about how to tackle a language. Good learners 
have available linguistic knowledge to cope with the complexities 
that may impede poor learners. This feature echoes Rubin’s good 
learner who is “prepared to attend to foim” (1975: 47). 

e. Strategies of experimentation and planning with the object of 
developing the new language into an ordered system, and revising 
this system progressively. Good learners are purposeful and 
systematic in developing the second language. 

f. Constantly searching for meaning. Good learners understand the 
cultural significance of language use, and make efforts to get not 
only surface meaning but also the underlying sociocultural 
connotations as well. 

g. Willingness to practice. Good learners know practice makes perfect 
and search for opportunities to practice the language. 

h. Willingness to use the language in real communication. Good 
learners seek every opportunity available to use the language, since 
they realize that formal practice does not transfer to functional use. 

i. Self-monitoring and critical sensitivity to language use. Good 
learners are sensitive and self-critical in language use. 

j . Developing the second language more and more as a separate 
reference system and learning to think in it. Good learners strive to 
attain an intuitive command of the language. 

Profile bv Naiman. Frohlich. and Todesco 

Naiman, Frohlich, and Todesco (1975), through interviews in a study designed 
to more accurately discover behaviors of successful language learner, provided a 
list of six characteristics related to good language learner. The list includes the 
following: 

a. Finding a set of individual learning preferences and selecting 
language situations that allow for those preferences to be used. 

b. Actively engaging in language learning processes through language 
activities, discovering and dealing with individual learning 
problems, and searching for opportunities to use the language. 

c. Enhancing an awareness about the language both as a rule system 
of grammar and as a means of communication. 

d. Constantly extending and revising one’s understanding of the target 
language system as new information is internalized. 

e. Learning to think in the language, and developing the second 
language into a reference system separate from that in the native 
language. 

f. Addressing the affective demands imposed on language learning. 
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Research efforts concentrating on the good language learner had managed to 
identify strategies used by students. Rubin (1981) proposed related lists of 
learning strategies, including deduction, inferencing, practicing, and so forth 
(Refer to Table 1). She introduced the distinction between direct and indirect 
strategies, with the direct strategies being immediately involved in learning (e.g., 
inferencing) and the indirect strategies contributing indirectly to language 
learning (e.g., creating opportunity for practice). 

With Rubin’s classification scheme that subsumes learning strategies under 
two primary groupings and a number of subgroups, 1970s’ LLS research 
generated an alternative classification scheme such as Naiman et al.’s (1978). 
This system contains five broad categories of learning strategies and a number of 
secondary subsets, as illustrated in Table 1. 

2. 1980s — Cognitive psychology-based research 
The major research in the 1980s was that conducted by O’Malley and his 
colleagues in the United States. The researchers found a research program on the 
cognitive information processing view of human learning. The research was 
based mainly on Anderson’s production systems (O’Malley and Chamot, 1990). 
This paper briefly summarizes the Anderson framework and then shows the 
relationship between cognitive theory and strategies. 

A. Summary of Anderson’s cognitive theory 
Anderson portrayed the human information processor in terms of three types of 
information storage: one short-term memory and two interrelated long-term 
memories, namely a declarative memory and a production memory. The basic 
framework is summed up by Figure 1. 

Anderson also posited that there are two kinds of knowledge: declarative 
knowledge and procedural knowledge. Declarative knowledge consists of all the 
facts the learner has learned and all the experiences the learner has had. In short, 
declarative knowledge involves what the learner knows. In contrast, procedural 
knowledge involves the learner's knowing how to do something. For instance, 
our abilities to understand language or apply rules to the solution of a problem 
would be examples of procedural knowledge. 
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Figure 1 

A general framework for the ACT (Adaptive Control of Thought) system, 
identifying the major structural components and their interlinking processes 



APPLICATION 



DECLARATIVE 




PRODUCTION 


MEMORY 




MEMORY 


STORAGE 




MATCH 


RETRIEVAL 


WORKING 

MEMORY 


EXECUTION 


ENCODING 




PERFORMANCES 



OUTSIDE WORLD 



Source: J. R. Anderson (1983: 19) 



In this context, all knowledge is initially declarative. When information that 
has been stored in the declarative memory is proceduralized into a behavior by 
means of interpretive strategies, notably general problem solving and analogy, 
the information is transferred to the production memory. This process involves a 
creation of productions, which are IF (condition) . . . THEN (action) statements. 
Anderson (1983: 5-6) claimed that 

. . . underlying human cognition is a set of condition-action pairs 
called productions. The condition specifies some data patterns, and if 
elements matching these patterns are in working memory, then the 
production can apply. The action specifies what to do in that state. The 
basic action is to add new data elements to working memory. 

As successful proceduralized productions are repeated, they will eventually 
become fully autonomous. Anderson, therefore, argued that there exist three 
stages of skill acquisition. The earliest stage of learning is a cognitive stage, in 
which only declarative knowledge can be accessed in performance. At the second 
stage, the so-called associative stage, learning progresses and productions are 
formed through processes of composition and proceduralization. The autonomous 
stage is the last at which time the information is stored in more economical 
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productions that enable the learner to access more rapidly the productions 
through matching. 

Adapting this theoretical notion to their strategy research, O’Malley and 
Chamot (1990) provided a description and classification of strategies. For them, 
language is a complex cognitive skill that can be described within the context of 
cognitive theory, and language learning strategies are “special ways of processing 
information that enhance comprehension, learning, or retention of the 
information” (1990: 1). O’Malley and Chamot, therefore, characterized strategies 
within the production system with an IF condition and a THEN action 
constituent, as illustrated below: 

IF the goal is to comprehend an oral or written text, and I am 
unable to identify a word’s meaning, 

THEN I will try to infer the meaning from context. 

IF the goal is to comprehend a concept in a written text, and I 
know the concept is not at the beginning, 

THEN I will scan through the text to locate the concept. 

(O’Malley and Chamot, 1990: 52) 

They argued that through these proceduralizations, learners can strengthen their 
L2. 

B. Study by O’Malley et al. 

O’Malley and his colleagues (1985a) conducted interviews with, and 
observations of, high-school ESL (English as a Second Language) students in 
typical language learning classrooms. The interviewers asked for strategy reports 
of the various activities, and the collected strategy data produced a well-known 
language learning strategy classification systems (Refer to Table 2). The system 
classified strategies into three broad categories, which are metacognitive, 
cognitive, and social mediating strategies. 

It is striking that many of the strategies by 1970s’ researchers, such as 
Naiman et al. and Wong-Fillmore, are excluded here. It is also noticeable that the 
work of O’Malley and his colleagues, which was based on cognitive theory, 
considerably enlarged the role of the learner as a self-reflective being with insight 
into his or her learning processes. As they put it. 

This line of research suggests that transfer of strategy training to new 
tasks can be maximized by pairing cognitive strategies with appropriate 
metacognitive strategies. Students without metacognitive approaches 
are essentially learners without direction or opportunity to review their 
progress, accomplishments, and future directions. 

(O’Malley, Chamot, Stewner-Manzares, Russo, and Kupper, 

1985b: 560-561) 
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This study (1985a) also found that both beginning and intermediate students 
used an extensive variety of learning strategies, and interestingly intermediate 
level students tended to use metacognitive strategies more often than beginners. 
Additionally, this study revealed that ESL teachers were generally unaware of 
students’ strategies and rarely introduced strategies while teaching. 

C. Chamot and Kupper’s study 

Another notable study based on cognitive theory was a three-year project by 
Chamot and Kupper (1989). In this study, involving group interviews and 
thinking-aloud protocols, Chamot and Kupper found that successful language 
learners use language learning strategies more effectively than do less successful 
language learners. According to the study, effective students were purposeful, 
monitored their performance globally, and used background knowledge 
successfully. Chamot and Kupper believed that although students of all ability 
levels use learning strategies, many language learners are low achievers simply 
because they rely on infrequent or inappropriate use of a narrow, limited 
repertoire of strategies, such as rote memorization, translation, repetition, and 
deduction. 



Table 2 

Learning Strategy Definitions (O’Malley et al., 1985b: 582-584) 



Learning Strategy Description 

METACOGNITIVE STRATEGIES 



Advance Organizers 

Directed Attention 
Selective Attention 

Self-Management 

Functional Planning 

Self-Monitoring 



Making a general but comprehensive preview of 
the organizing concept or principle in an 
anticipated learning activity 
Deciding in advance to attend in general to a 
earning task and to ignore irrelevant distracters 
Deciding in advance to attend to specific aspects 
of language input or situational details that will 
cue the retention of language input 
Understanding the conditions that help one learn 
and arranging for the presence of those 
conditions 

Planning for and rehearsing linguistic 
components necessary to carry out an upcoming 
language task 

Correcting one’s speech for accuracy in 
pronunciation, grammar, vocabulary, or for 
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appropriateness related to the setting or to the 
people who are present 

Delayed Production Consciously deciding to postpone speaking in 

order to learn initially through listening 
comprehension 

Self-Evaluation Checking the outcomes of one’s own language 

learning against an internal measure of 
completeness and accuracy 



COGNITIVE STRATEGIES 



Repetition 

Resourcing 

Translation 

Grouping 

Note Taking 



Initiating a language model, including overt 
practice and silent rehearsal 
Using target language reference materials 
Using the first language as a base for 
understanding and/or producing the second 
language 

Reordering or reclassifying, and perhaps 
labeling, the material to be learned, based on 
common attributes 

Writing down the main idea, important points, 
outline, or summary of information presented 
orally or in writing 



COGNITIVE STRATEGIES 

Deduction Consciously applying rules to produce or 

understand the second language 

Recombination Constructing a meaningful sentence or larger 

language sequence by combining known 
elements in a new way 

Imagery Relating new information to visual concepts in 

memory via familiar, easily retrievable 
visualizations phrases, or locations 

Auditory Representation Retention of the second or a similar sound for a 

word, phrase, or longer language sequence 

Keyword Remembering a new word in the second 

language by (1) identifying a familiar word in 
the first language that sounds like or otherwise 
resembles the new word and (2) generating 
easily recalled images of some relationship 
between the new word and the familiar word 
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Contextualization Placing a word or phrase in a meaningful 

language sequence 

Elaboration Relating new information to other concepts in 

memory 

Transfer Using previously acquired linguistic and/or 

conceptual knowledge to facilitate a new 
language learning task 

Inferencing Using available information to guess meanings 

of new items, predict outcomes, or fill in 
missing information 



SOCIOAFFECTIVE STRATEGIES 



Cooperation Working with one or more peers to obtain 

feedback, pool information, or model a language 
activity 

Question for Clarification Asking a teacher or other native speaker for 

repetition, paraphrasing, explanation, and/or 
examples 



3. 1990s ™ Vygotskian approach to second language learning 
strategy research 

Recently, there has been a growing interest in the approach to cognition 
developed principally by Lev S. Vygotsky, which offers a fundamentally 
different view from information processing theory and has found application in 
the study of SLA (Second Language Acquisition) as well. LLS research in 1990s 
also reveals the impact from Vygotskian theory, and therefore, it is necessary to 
review Vygotsky’s basic ideas, so as to understand the direction of the study of 
LLS in the current literature. 



A. The Vygotskian perspective 

In order to trace the behavioral change of human development, Vygotsky was 
interested in how children overcome the difficulties that they confront. Thus 
Vygotsky observed the phenomenon of a little child invoking private speech 
while trying to solve a problem. From such observations and using the 
theoretical framework of dialectical and historical materialism, Vygotsky 
explained transformation of elementary psychological processes into higher ones. 

Vygotsky believed that culturally provided sign systems such as speech or 
language function as the “mediating link’’ between the individual and the world, 
and the mediating link ultimately becomes the bridge between early and later 
forms of individual development (Vygotsky, 1978). Therefore, the term “speech’’ 
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reflects an important fact that Vygotsky viewed individual mental functioning as 
deriving essentially from the internalization of social processes. Such a belief, 
accordingly, led Vygotsky to assert that the child’s psychological development 
appears first between the child and other persons and then within the child 
(Vygotsky, 1986; Wertsch and Tulviste, 1992). The following sample dialogues 
demonstrate well what Vygotsky means: 

Stage one : Interacting with adults or better speakers 

Brenda* hiding 

Adult: Hiding? What’s hiding? 

Brenda balloon 

Takahiro: This 

broken 

Adult: Broken. 

Takahiro: broken 

This /dz/ broken, 
broken 

Adult: Upside down. 

Takahiro: upside down 

this broken 
upside down 
broken. 

Source: Peck (1978: 385) 

Stage Two : Interacting with self 

A four-and-a-half-year-old girl was asked to get candy from a cupboard 
with a stool and a stick as tools. 

Girl: (Stands on a stool, quietly looking, feeling along a 

shelf with stick.) On the stool. (Glances at 
experimenter. Puts tick in other hand.) Is that really 
the candy? (Hesitates.) No, that doesn’t get it. I 
could use the stick. (Takes stick, knocks at the 
candy.) It will move now. (Knocks candy.) It 
moved, I couldn’t get it with the stool, but the stick 
worked. 

Source: Vygotsky (1978: 25) 

Stage Three : Internalization, automatization 
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Noriko: (without help from others or self-talk) Mi-Ae, can I 

borrow your cassette tapes, Whaddaya say, tonight? 

Given the scheme of human development, Vygotsky proposed the notion of 
the Zone of Proximal Development (ZPD). In his words, this zone constitutes 

... the distance between the actual developmental level as determined by 
independent problem-solving and the level of potential development as 
determined through problem-solving under adult guidance or in 
collaboration with more capable peers. 

(Vygotsky, 1978: 86) 

The actual developmental level characterizes the learner’s ability to carry out 
tasks without help, while the potential developmental level characterizes the 
learner’s ability to perform with the help of an adult or expert. For Vygotsky, 
effective learning or teaching tasks takes place only in the ZPD. By the notion 
of ZPD, Vygotsky again emphasized that higher cognitive processes emerge as a 
result of interactions between people. In sum, according to the Vygotskian 
perspective, not only does the learner actively construct knowledge, but the 
individual’s acquisition suggests socially mediated activity. 

A number of researchers have attempted to apply the Vygotskian view of 
human learning to LLS studies. Oxford and colleagues have explored mainly the 
relation between learning styles and strategies, the so-called style-strategy 
linkage. On the other hand, another group of scholars, including Lantolf and 
Appel (1994) and Donato and McCormick (1994), have made sociocultural 
claims in the area. 

B. Style-strategy linkage 

In 1990s, one of the most productive researchers on the LLS issue has been 
Oxford. Oxford and her colleagues, in a number of empirical studies (Ehrman and 
Oxford, 1989, 1990; Oxford and Nyikos, 1989; etc.), have found a statistical 
link between students’ L2 learning strategies and learning styles. Briefly put, 
learning styles are the general approaches to learning or problem-solving, while 
learning strategies are “the specific behaviors or actions — often conscious — used 
by the students to improve or enhance their learning process” (Oxford et al., 
1992: 440). 

According to numerous studies about language learning styles and strategies 
(Ehrman and Oxford, 1990; Hansen and Stansfield, 1981; Jamieson, 1992; 
Oxford et al., 1992; Oxford, 1993; Oxford and Ehrman, 1993), individual 
learners seem to have a composite of the following style-dimensions that are 
regarded as the most significant for language learning: global and analytic; field- 
dependent and field-independent; visual, auditory, and hands-on, etc. For instance, 
a well-known study by Ehrman and Oxford (1990) found regularities in the 
preferred learning strategies associated with each of eight psychological type 
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preferences by the language learners: The eight psychological types are formed 
by combinations of dimensions, one each from the pairs of sensing/intuition, 
thinking/feeling, introversion/extroversion, and judging/perceiving. 

A key assumption here is that learning styles and learning strategies have a 
strong cultural component. Accordingly, Oxford et al. (1993) attempted to show 
cross-cultural style conflicts in EFL (English as a Foreign Language) 
classrooms, and meanwhile Reid (1987) concluded that ESL students from 
different language backgrounds differ from one another in their learning style 
preferences. Therefore, learning styles, a significant determiner of L2 strategy 
choice, seem to be very much influenced by the individual’s sociocultural 
background. Applying Vygotsky’s work to SLA, Scarcella and Oxford (1992) 
implied that language learners can and must extend themselves beyond their 
“stylistic comfort zone” and use new behaviors with the teacher’s assistance (p. 
63). 

Oxford’s notable contribution to LLS research is that she proposed a new 
strategy classification system after reviewing the characteristics of language 
learning strategies. As shown below (Refer to Figure 2), Oxford divided learning 
strategies into six branches: 1) memory, 2) cognitive, 3) compensation, 4) 
metacognitive, 5) affective, and 6) social strategies. 

Donato and McCormick (1994), however, strongly argued that the Vygotskian 
method differs from the current research on LLS which relies heavily on “making 
lists” of particular aspects of an individual’s strategy use. According to Donato 
and McCormick, this cultural-historical approach “goes beyond static 
taxonomies” (1994: 454). Nevertheless, Adair-Hauck and Donato (1994) regarded 
Oxford as one of the Vygotskian second language specialists, because she has 
invoked the theories of Vygotsky, especially, in terms of the concept of ZPD 
and the emphasis on social context supports in language learning. 

One who observes a gradual shift in mood from Oxford’s earlier studies 
through current ones may see Oxford’s theories as a transition between the 
1980s’ cognitive approach to LLS and the 1990s’ Vygotskian perspective. 

Figure 2 



Diagram of the strategy System Showing Two Classes, Six Groups, and 19 Sets 



DIRECT STRATEGIES 
1. Memory and strategies 

A. Creating mental linkages 

B. Applying images and sounds 

C. Reviewing well 

D. Employing action 



INDIRECT STRATEGIES 
I. Metacognitive strategies 

A. Centering your learning 

B. Arranging and planning your 

learning 

C. Evaluating your learning 
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II. Cognitive strategies 

A. Practicing 

B. Receiving and sending 
messages 

C. Analyzing and reasoning 

D. Creating structure for 
input and output 

III. Compensation strategies 

A. Guessing intelligently 

B. Overcoming limitations 
in speaking and writing 



II. Affective strategies 

A. Lowering your anxiety 

B. Encouraging yourself 

C. Taking your emotional 
temperature 



III. Social strategies 

A. Asking questions 

B. Cooperating with others 

C. Empathizing with others 



(Oxford, 1990: 17) 

C. Sociocultural theory of LLS research 

Sociocultural theorists define language learning strategies as instrumental 
actions to fulfill specific goals for learning (Donato and McCormick, 1994; 
Lantolf, 1994). In light of this theory, language learning strategies come from 
the primary social practice of the classroom. For instance, if the language class 
is too much focused on examinations or grading rather than on functional 
language practice, as is often the case in EFL, the use of strategies to develop 
communicative competence will be rare. Therefore, an LLS is “a by-product of 
the mediational processes” in a foreign language task (Lantolf, 1994: 419). In 
other words, to those theorists, social interaction and cultural institutions such 
as schools, classrooms, literature, etc. play important roles in an individual’s 
cognitive development. 

The researchers also suggested that activity theory is germane to the study of 
LLS. According to this claim, because activity theory defines activity as 
containing contextual, intentional, and circumstantial dimensions, activity 
theory enables the literature to adequately analyze LLS, that is, why the learner 
is using a particular strategy, how the learner performs the task, and how the 
situation shapes such strategic actions (Lantolf and Appel, 1994: 21). 

Unlike Oxford and her colleagues, sociocultural theorists believe that LLS 
cannot be directly taught, nor are they a function of cognitive style or 
individual’s personality (Donato and McCormick, 1994; Lantolf, 1994 : 419). 
Donato and McCormick (1994) examined portfolios that required students to 
supply concrete, self-selected evidence of their growing language abilities. The 
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portfolios thus differed from previous methodologies, such as diaries or journals. 
The researchers confirmed that strategies developed and changed over time, 
depending on the classroom culture. In this light, sociocultural theorists tend to 
pay acute attention to processes and changes, not to products, as Schinke-Llano 
(1993) has pointed out. 

In fact, the perspective of sociocultural theory in LLS study seems to be in its 
infancy. As a result, there is a need for further research to examine its claims. 

Intending to be dedicated to a general understanding of LLS research, the 
present paper provided a brief summary of the trends of LLS research from the 
1970s to the mid-1990s in terms of their theoretical backgrounds. The author 
would like to hear further discussions on a more explanatory framework for LLS 
research. 
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